Failed validation: XsdFieldSelector(path='@trigger|@attime') field selects multiple values

[aardalath]

Hi,

I have an XML with events defined that can occur when a certain trigger from a third system arrives or at a given fixed time. For that, we wanted to have a single event element type definition with several attributes, two of them, trigger or attime, mutually exclusive. In order to implement that I found this solution:

. . .
      <xs:element type="eventType" name="event" maxOccurs="unbounded" minOccurs="0">
        <xs:key name="attributeKey">
          <xs:selector xpath="."/>
          <xs:field xpath="@trigger|@attime"/>
        </xs:key>
      </xs:element>
. . .

So, I created a file compliant with this schema, with event nodes having either trigger or attime attribute, but not both, and another one with a failing instance of event, with trigger and attime at the same time.

The problem is that both failed, and XMLSchema.validate() raises in both cases an exception for a valid event instance (with only the trigger attribute), as follows:

Traceback (most recent call last):
  File "/home/user/venv/lib/python3.10/site-packages/xmlschema/validators/elements.py", line 911, in collect_key_fields
    xsd_fields = identity.get_fields(xsd_element.xpath_node)
  File "/home/user/venv/lib/python3.10/site-packages/xmlschema/validators/identities.py", line 331, in get_fields
    raise XMLSchemaValueError(msg % field)
xmlschema.exceptions.XMLSchemaValueError: XsdFieldSelector(path='@trigger|@attime') field selects multiple values!

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/user/trigev/etc/trigev/schema/./validate.py", line 12, in <module>
    raise(ee)
  File "/home/user/trigev/etc/trigev/schema/./validate.py", line 9, in <module>
    isvalid = schema.validate(sys.argv[2])
  File "/home/user/venv/lib/python3.10/site-packages/xmlschema/validators/schemas.py", line 1698, in validate
    for error in self.iter_errors(source, path, schema_path, use_defaults,
  File "/home/user/venv/lib/python3.10/site-packages/xmlschema/validators/schemas.py", line 1829, in iter_errors
    for result in xsd_element.iter_decode(elem, **kwargs):
  File "/home/user/venv/lib/python3.10/site-packages/xmlschema/validators/elements.py", line 739, in iter_decode
    for result in content_decoder.iter_decode(obj, validation, **kwargs):
  File "/home/user/venv/lib/python3.10/site-packages/xmlschema/validators/groups.py", line 1050, in iter_decode
    for result in xsd_element.iter_decode(child, validation, **kwargs):
  File "/home/user/venv/lib/python3.10/site-packages/xmlschema/validators/elements.py", line 739, in iter_decode
    for result in content_decoder.iter_decode(obj, validation, **kwargs):
  File "/home/user/venv/lib/python3.10/site-packages/xmlschema/validators/groups.py", line 1050, in iter_decode
    for result in xsd_element.iter_decode(child, validation, **kwargs):
  File "/home/user/venv/lib/python3.10/site-packages/xmlschema/validators/elements.py", line 836, in iter_decode
    yield from self.collect_key_fields(obj, xsd_type, validation, nilled, **kwargs)
  File "/home/user/venv/lib/python3.10/site-packages/xmlschema/validators/elements.py", line 922, in collect_key_fields
    yield self.validation_error(validation, err, obj, **kwargs)
  File "/home/user/venv/lib/python3.10/site-packages/xmlschema/validators/xsdbase.py", line 249, in validation_error
    raise error
xmlschema.validators.exceptions.XMLSchemaValidationError: failed validating <Element 'event' at 0x71e1d337c040> with XsdElement(name='event', occurs=[0, None]):

Reason: XsdFieldSelector(path='@trigger|@attime') field selects multiple values!

Schema component:

  <xs:element xmlns:xs="http://www.w3.org/2001/XMLSchema" type="eventType" name="event" maxOccurs="unbounded" minOccurs="0">
    <xs:key name="attributeKey">
      <xs:selector xpath="." />
      <xs:field xpath="@trigger|@attime" />
    </xs:key>
  </xs:element>

Instance type: <class 'xml.etree.ElementTree.Element'>

Instance:

  <event trigger="TS01" command="powu56.py" parameters="-on -ports '1,2,3'" />

Path: /mqttservices/events/event[1]

I checked everything with xmllint and in that case everything works as expected, that is, the valid XML file validates, and the failing one fails to validate, and the correct reason is provided. But, of course, I would like to use a Python solution like xmlschema.

Please, find attached the sample XML and XSD files, as well as a very simplistic Python script.

test_keys.zip

[brunato]

Hi,
I've took time for checking your test files with other validators. The results with online XSD validator at freeformatter.com gives both instances as valid. The same results with the latest version of Xerces J 2.

So I checked the XSD 1.1 section on Identity constraints, that contains a paragraph about key fields:

{fields} specifies XPath expressions relative to each element selected by a {selector}. Each XPath expression in the {fields} property must identify a single node (element or attribute), whose content or value, which must be of a simple type, is used in the constraint. It is possible to specify an ordered list of {fields}s, to cater to multi-field keys, keyrefs, and uniqueness constraints.

In order to reduce the burden on implementers, in particular implementers of streaming processors, only restricted subsets of XPath expressions are allowed in {selector} and {fields}. The details are given in Constraints on Identity-constraint Definition Schema Components (§3.11.6).
Note: Provision for multi-field keys etc. goes beyond what is supported by xsl:key.
Note: In version 1.0 of this specification [XSD 1.0 2E], identity constraints used [XPath 1.0]. They now use [XPath 2.0].

This is still not completely clear to me, so I will check other XSD W3C tests of similar cases to decide if a patch is needed, and which.

About the results of xmllint (based on the libxml2's XSD validator) if the key constraint is effective this should be clearly invalid, but is reported as valid:

 xmllint --schema test_keys.xsd test_keys-invalid.xml 
<?xml version="1.0" encoding="utf-8"?>
<mqttservices active="true">
  <events>
    <event trigger="TS01" command="powu56.py" parameters="-on -ports '1,2,3'"/>
    <event trigger="TS08" command="powu56.py" parameters="-off -ports '1,2,3'"/>
    <event trigger="TS08" command="powu56.py" parameters="-off -ports '1,2,3'"/>
    <event attime="2:00" command="powu56.py" parameters="-off -ports '1,2,3'"/>
    <event attime="2:00" command="powu56.py" parameters="-off -ports '1,2,3'"/>
  </events>
</mqttservices>
test_keys-invalid.xml validates

then I don't trust the results of libxml2/xmllint on this, Xerces J2 is a more compliant validator (with full XSD 1.1 support, libxml2 support only XSD 1.0), but I've to understand which node-set is intended in case of multiple members.

thank you

[brunato]

After checking the related XSD W3C tests it seems that something is not aligned also for Xerces that not reports the test "idL100.v" as invalid. Initially the test instance was considered valid, but it changed to invalid after an issue on it:

• Schema: idL100.xsd
• Instance: idL100.xml
• Bug ticket: http://www.w3.org/Bugs/Public/show_bug.cgi?id=4061

So I going to change xmlschema to consider only XML instance values for checking the duplicated values. This should permits to build the XSD fields once, ignoring duplicated XSD nodes-decoders (but the fields decoders must be changed admitting a also a list of decoders and then an intermediate matching have to be done). With these modifications the XML samples that you provided will be validated as you expected.

What I ask you is if the test_keys-invalid.xml above has to be consider valid or not. If we don't agree on invalidity I need an explanation that helps me on how to handle these cases in the code.

thank you

[brunato]

Note: my error, test_keys-invalid.xml is valid, the key is defined for <event> not for <events>.

Reworked identities decoding, using XPath typed selectors, the fix will be available with next release.

thank you

[brunato]

Version v3.4.3 fix identity validation with multiple values. Added your example to the test cases.
thank you