question: how to enumerate annotations

[redtide]

Hello, I would like to access all the xs:documentation content to export for translation, but I wasn't able to.
From the read the docs I see in the API there is a XsdAnnotation class that owns both appinfo and documentation.
I tried also with plain xml.etree.ElementTree to parse the file without success.

[brunato]

Hi,
you have to iterate components and access their annotation property (annotations are build lazy because they are not needed for validation):

>>> import xmlschema
>>> schema = xmlschema.XMLSchema("<your schema source>")
>>>
>>> for c in schema.iter_components(xsd_classes=xmlschema.XsdComponent):
...    if c.annotation is not None:
...       print(c.annotation.appinfo)
...       print(c.annotation.documentation)

For iterating over all loaded schemas you can use:

>>> for c in schema.maps.iter_components(xsd_classes=xmlschema.XsdComponent):
...    if c.annotation is not None:
...       print(c.annotation.appinfo)
...       print(c.annotation.documentation)

Note: currently the annotation property is not defined for schema objects, this is why I use the filter xsd_classes=xmlschema.XsdComponent. I will add this property soon also for schemas.

[redtide]

Grazie mille per la risposta rapida!

Unfortunately I still have an issue: I forgot to mention that (beside I'm new to both xsd and xmlschema) I had to use v1.1 (which leaded me to use your library) because I heard that it's the version that permits more than 1 maxOccurs as element attribute value in a all tag, 3 in this case, and it works as expected using XMLSchema11 for validation.

So following your example I came up with a

    schema = xmlschema.XMLSchema11(args.schema)
    for c in schema.iter_components(xsd_classes=xmlschema.XsdComponent):
      if c.annotation is not None:
        for item in c.annotation.documentation:
            print(item.text)

but I get an error XMLSchemaParseError - attribute 'maxOccurs' not allowed anyway.
Removing it I noticed that some items are repeated and in case of a restriction, only the first item appears:

expand

Active window title
Defines some general behavior of windows
Defines some general behavior of windows
Defines the number and names of desktops
Defines aspects of window focus
Defines desktop margins
Defines desktop margins
Defines behaviour of windows when cursor close to the usable area
Show a small indicator on top of the window when resizing or moving
Show a small indicator on top of the window when resizing or moving

[brunato]

I don't understand where the XMLSchemaParseError is raised. In XSD 1.1 the all groups still have maxOccurs 0 or 1, but can contain other groups.

The repetitions are originated by implicit internals attribute groups that share the same elem with correspondant complex types. It should be fixed with a redefinition of the property that parse the annotation.

Thank you

[redtide]

I don't understand where the XMLSchemaParseError is raised. In XSD 1.1 the all groups still have maxOccurs 0 or 1, but can contain other groups.

isn't because of my wrong maxOccurs="3" in the highlighted line?
I wonder why it gets validated when using schema.is_valid().

So I can convert the resulting list to a set as workaround, but now I wonder also how I can permit the font element to be repeated 3 times with 3 different attributes ( I don't even know if this is correct in XML, though it works).

[brunato]

isn't because of my wrong maxOccurs="3" in the highlighted line?
I wonder why it gets validated when using schema.is_valid().

anyway maxOccurs="3" is admitted with XSD 1.1. For clarity this is the schema parsing results for me:

>>> import xmlschema
>>> schema = xmlschema.XMLSchema11('rc.xsd')
>>> schema = xmlschema.XMLSchema('rc.xsd')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/brunato/Development/projects/xmlschema/xmlschema/validators/schemas.py", line 464, in __init__
    self.parse_error(e.reason or e, elem=e.elem)
  File "/home/brunato/Development/projects/xmlschema/xmlschema/validators/xsdbase.py", line 190, in parse_error
    raise error
xmlschema.validators.exceptions.XMLSchemaParseError: attribute maxOccurs='3': value must be one of [0, 1]:

Schema:

  <xs:element xmlns:xs="http://www.w3.org/2001/XMLSchema" name="font" type="labwc:font" minOccurs="0" maxOccurs="3" />

Path: /xs:schema/xs:complexType[13]/xs:all/xs:element[4]

So I can convert the resulting list to a set as workaround, but now I wonder also how I can permit the font element to be repeated 3 times with 3 different attributes ( I don't even know if this is correct in XML, though it works).

Using sets is correct with translation strings, because there could be the same description in different components.
Anyway I'll try to reduce duplication of annotations from implicit internal components (a sample XSD case could help to build a test for checking issues on this ...).

[redtide]

First of all, thank you very much to work on this!

anyway maxOccurs="3" is admitted with XSD 1.1.

So the syntax is correct? Thank you for the confirmation!

Using sets is correct with translation strings, because there could be the same description in different components.

Right!

Anyway I'll try to reduce duplication of annotations from implicit internal components (a sample XSD case could help to build a test for checking issues on this ...).

Nice!
As a side note: using the maps global recursion alternative method, spits out a lot of internal docs, just to mention now also as self reminder.

Removing it I noticed that some items are repeated and in case of a restriction, only the first item appears

I came out with a working Qt translation for XSD, the only issue remains is the underlined one:

  <xs:simpleType name="fontplace">
    <xs:restriction base="xs:string">
      <xs:enumeration value="ActiveWindow">
        <xs:annotation><xs:documentation>Titlebar of active window</xs:documentation></xs:annotation>
      </xs:enumeration>
      <xs:enumeration value="MenuItem">
        <xs:annotation><xs:documentation>Menu item label (currently only root menu)</xs:documentation></xs:annotation>
      </xs:enumeration>
      <xs:enumeration value="OnScreenDisplay">
        <xs:annotation><xs:documentation>Items in the on screen display</xs:documentation></xs:annotation>
      </xs:enumeration>
    </xs:restriction>
  </xs:simpleType>

finds only Titlebar of active window. I wonder if I need to make another iteration with a different element.

Let me know if there is something to open a single issue for, for a later resolution time and avoid confusion in this one.

[brunato]

As a side note: using the maps global recursion alternative method, spits out a lot of internal docs, just to mention now also as self reminder.

This because it accesses also to XSD meta-schema annotations. Maybe there should be a specific method for extracting all annotations, with various filtering choices.

finds only Titlebar of active window. I wonder if I need to make another iteration with a different element.

There are also other cases to consider. For example in this chunk:

  <xs:simpleType name="fontplace">
    <xs:restriction base="xs:string">
      <xs:enumeration value="ActiveWindow">
        <xs:annotation><xs:documentation>Titlebar of active window</xs:documentation></xs:annotation>
      </xs:enumeration>
      <xs:enumeration value="MenuItem">
        <xs:annotation><xs:documentation>Menu item label (currently only root menu)</xs:documentation></xs:annotation>
      </xs:enumeration>
      <xs:enumeration value="OnScreenDisplay">
        <xs:annotation><xs:documentation>Items in the on screen display</xs:documentation></xs:annotation>
      </xs:enumeration>
    </xs:restriction>
  </xs:simpleType>

there could be annotations also on xs:simpleType and xs:restriction elements: how can be these considered and associated with?

Let me know if there is something to open a single issue for, for a later resolution time and avoid confusion in this one.

It could help a PR contribution with a full annotated XSD schema (or several smaller fully annotated schemas) that can be used for building tests an then to fix the code.

[redtide]

This because it accesses also to XSD meta-schema annotations. Maybe there should be a specific method for extracting all annotations, with various filtering choices.

I don't know much well the system yet, I only expect that an user xsd should be parsed to extract only his data, not the "system' libraries" one

there could be annotations also on xs:simpleType and xs:restriction elements: how can be these considered and associated with?

to me the question is, in other words respect what I said above, how can I retrieve only the data I specified in my schema file/namespace?

It could help a PR contribution with a full annotated XSD schema (or several smaller fully annotated schemas) that can be used for building tests an then to fix the code.

that sounds a simple task I can do, but I have no idea ATM about which specific cases I should reproduce

[brunato]

I don't know much well the system yet, I only expect that an user xsd should be parsed to extract only his data, not the "system' libraries" one

Meta-schema is not a library but only a set of base schemas for validating XSD sources.

to me the question is, in other words respect what I said above, how can I retrieve only the data I specified in my schema file/namespace?

you can iterate all the schemas of the global maps instance (using iter_schemas()) and filter them on the schema.target_namespace value.

that sounds a simple task I can do, but I have no idea ATM about which specific cases I should reproduce

a case that summarizes the annotations of your effective XML case(s) maybe sufficient

[redtide]

Sorry for the delay.
The example above basically? I tried to adapt this to your test but I'm not sure about the entire process (I mean if I did something wrong in the way), so I made a simplified case in the attachment.

The script works by cd in the test-annotations directory and it will write the results in the src/annotations.hpp file replacing the string in the other pseudo template file; there it can be seen that only the first of the three annotations are extracted.

test-annotations.tar.gz

[brunato]

Hi,
i will include your enumeration case for extending and sharpening annotations processing. This can be planned for a minor release of version 3 of the package.

[redtide]

Hello, OK, thank you!

[brunato]

Hi,
the v3.1.0 has a resolution for this. Each component and the schema have a property called annotations that returns a list with all the annotations detected for it.
The detection is made with a simple criteria, starting from the Element (elem) related to the component down to descendant elements that are not related with other components. This should avoid both repetitions and unparsed annotations.

Try that and report me any problem on using the new feature.

Thank you

[redtide]

Thank you for the work! I'll take a bit to finish some projects work, I'll try and let you know if solved ASAP.