Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

subtract/exclude/remove from, for applicability? #177

Closed
ay-ex opened this issue Jul 27, 2023 · 18 comments
Closed

subtract/exclude/remove from, for applicability? #177

ay-ex opened this issue Jul 27, 2023 · 18 comments
Milestone

Comments

@ay-ex
Copy link

ay-ex commented Jul 27, 2023

First of thank you for the amazing IDS mechanics, they are great for doing checks on IFC model quality. 🙂
One thing I was missing (or maybe just overlooked?) when reading through documentation and github issues
while creating rules for some of our projects, is the ability to subtract/exclude/remove something from applicability.

E.g.:
We would like to set the applicability to all elements of certain Ifc classes with a specific Pset.Property value,
then excluding the ones with a specific Name.
But the applicability steps seems to only append not subtract.(?)

Ideally it would possible to do the following:

  • Applicability 1:
    • Add all elements of
      • IfcWall, IfcDoor, IfcWindow, IfcBuildingElementProxy
        • with a specific "TestPset.TestProp"
          • with value "Foo".
  • Applicability 2:
    • Remove from existing applicability all
      • IfcBuildingElementProxy
        • with "Attribute" "Name"
          • with regex value: "Pyramid.*"

Is this currently (0.9) achievable with IDS?
Thank you for clarification and/or any pointers.

@NickNisbet
Copy link

NickNisbet commented Jul 27, 2023 via email

@NickNisbet
Copy link

NickNisbet commented Jul 29, 2023 via email

@berlotti
Copy link
Member

The current XSD pattern is not supporting this.
There is a plan to migrate to a more advanced regex dialect, but that is on the roadmap for 1.x or 2.x

@andyward
Copy link
Contributor

I can see these kinds of 'composite' requirements will need some extra schema support - if we want to handle all the AND, OR, NOT boolean permutations in a future release... Presumably some kind of Grouping and Operator support in the schema - I'm not sure how a regex would cope!?

Just on Nick's query on 0.9 schema, I've wondered the same thing:

Can I check if this is wrong? I expected that each entry in an Applicability list would progressively reducing the number of relevant entries. The message below suggests that it is acting as a Selection, progressively increasing the number of relevant entries.
Similarly I would expect that that each entry in an Requirement list would progressively make separate checks on relevant entries.

My reading was that within a Requirements list the facets do refine. i.e. there's an implicit AND between facets on requirements. That seems to be the intent expressed in this test case fail-a_specification_passes_only_if_all_requirements_pass_1_2.ids i.e. For all Walls, the Name must be 'Waldo' AND the Description must be 'FooBar'

So all checks have to be met on requirement facets to satisfy the overall requirement.

On applicability lists, I couldn't find a test anywhere but my expectation is that multiple Applicability facets also refine the filter so more facets will produce fewer applicable matches (or never more). That's how we implemented in xbim, and seems in keeping with the examples in the developer documentation (e.g. "External load bearing walls need to have a fire rating property for code compliance" is clearly an AND in the Applicable facet)

Where is the expected behaviour documented and is it clear enough?

For me, the Requirement behaviour seems generally well documented in the test cases. By comparison, the Applicability behaviour seems much less well defined. I'll open a separate issue for this - Feels like we need to get that right before we consider these more complex specifications!

@andyward
Copy link
Contributor

Back to the original question, how to implement:

  • Applicability 1:
    * Add all elements of
    * IfcWall, IfcDoor, IfcWindow, IfcBuildingElementProxy
    * with a specific "TestPset.TestProp"
    * with value "Foo".
  • Applicability 2:
    * Remove from existing applicability all
    * IfcBuildingElementProxy
    * with "Attribute" "Name"
    * with regex value: "Pyramid.*"

... something like this might get close, but I'm not clear if cardinality can be used like that on Applicability facets:

      <specification>
	   <applicability>
                <entity>
                    <name>
                        <simpleValue>IFCBUILDINGELEMENTPROXY</simpleValue>
                    </name>
                </entity>
		<property datatype="IfcLabel">
                    <propertySet>
                        <simpleValue>TestPset</simpleValue>
                    </propertySet>
                    <name>
                        <simpleValue>TestProp</simpleValue>
                    </name>
		    <value>
                        <simpleValue>Foo</simpleValue>
                    </value>
                </property>
		<property minOccurs="0" maxOccurs="0"> /* is this valid? */
                    <name>
                        <simpleValue>Name</simpleValue>
                    </name>
                    <value>
                        <xs:restriction base="xs:string">
                            <xs:pattern value="Pyramid.*" />
                        </xs:restriction>
                    </value>
                </property>
            </applicability>
/* and duplicate spec for Walls,Doors,Windows */

@ay-ex
Copy link
Author

ay-ex commented Aug 15, 2023

@berlotti Thank you for confirming it is currently not available.
Similar to @andyward I fear extending regex (albeit welcome) will still not help in that use case.
(I tried it with an abomination of a limited regex negating the name (which works in isolation),
but that still does not seem to get me the AND connection between applicabilities @andyward mentions.)

@andyward Thank you for the example and comments - fully agree.
An AND (not only inside one facet as described by you) between applicabilities would be very helpful in this situation.

@berlotti
Copy link
Member

can't you split this into two specifications?

Specification 1:

  • Add all elements of
  • IfcWall, IfcDoor, IfcWindow, IfcBuildingElementProxy
  • with a specific "TestPset.TestProp"
  • with value "Foo".

Specification 2:

  • IfcBuildingElementProxy
  • with "Attribute" "Name"
  • with regex value: "Pyramid.*"
  • maxOccurs 0

?

@ay-ex
Copy link
Author

ay-ex commented Aug 24, 2023

@berlotti thank you for your suggestion.
I would fear the answer is no:
If these were two separate specifications,
we would get our IfcBuildingElementProxy 0-point-marker pyramid elements
(which are in the model, but should not be tested for Foo)
rightly flagged as objects, not fulfilling the TestPset.TestProp requirements.

Ideally we would be able to exclude the Pyramids from the applicability of specification 1.
(Or is maxOccurs 0 working as exclusion for specification1?)

@berlotti
Copy link
Member

maxoccurs can work on the specification:

image

@ay-ex
Copy link
Author

ay-ex commented Aug 24, 2023

@berlotti Thank you so much for the explanation!
So if I am reading this right, spec1 and spec2 work together to form the exclusion of pyramids via MaxOccors.
I have to try this once I am back on my work machine - I will give feedback. 🙂

@ay-ex
Copy link
Author

ay-ex commented Aug 28, 2023

@berlotti I tried your approach with two specifications as I understood it from your suggestion with this ids:
gh_exclusion_test_2specs.ids.txt
but it seems the restriction of spec2 has no effect on spec1: I get the same result for spec1 if I leave spec2 out. 🤔
Unfortunately spec1 still has the undesired behavior, that the pyramids IfcBuildingElementProxy are not excluded from the TestPset.TestProp check and therefore get flagged.

@ay-ex
Copy link
Author

ay-ex commented Aug 28, 2023

@andyward thank you for your suggested approach,
unfortunately that triggers the following error:

xmlschema.validators.exceptions.XMLSchemaValidationError: failed validating {'minOccurs': '0', 'maxOccurs': '0'} with XsdAttributeGroup():
Reason: 'minOccurs' attribute not allowed for element

which you seemed to have partly expected, judging by your comment.

@andyward
Copy link
Contributor

@ay-ex assuming this is ifcOpenShell's validator (which is not my area of expertise)? Try removing the minOccurs attribute from the property specification. I was writing the xml by hand, and minOccurs:0 is redundant when maxOccurs = 0.

i.e.

<property maxOccurs="0"> 

@ay-ex
Copy link
Author

ay-ex commented Aug 28, 2023

@andyward it is indeed.
With minOccurs removed, it starts to complain about maxOccurs:

xmlschema.validators.exceptions.XMLSchemaValidationError: failed validating {'maxOccurs': '0'} with XsdAttributeGroup():
Reason: 'maxOccurs' attribute not allowed for element

@berlotti
Copy link
Member

@berlotti I tried your approach with two specifications as I understood it from your suggestion with this ids: gh_exclusion_test_2specs.ids.txt but it seems the restriction of spec2 has no effect on spec1: I get the same result for spec1 if I leave spec2 out. 🤔 Unfortunately spec1 still has the undesired behavior, that the pyramids IfcBuildingElementProxy are not excluded from the TestPset.TestProp check and therefore get flagged.

The specifications are independent (this makes it possible to select them from a library).
Your second specification says there should not be any proxy elements with the pyramid pattern as the name.

@ay-ex
Copy link
Author

ay-ex commented Aug 29, 2023

@berlotti thank you for the explanation, then it is as I initially assumed.
But then I guess two specifications would not have helped me for the case:

I have a model with ~1000 elements of
IfcWall, IfcDoor, IfcWindow, IfcBuildingElementProxy
6 of them are pyramids which I would ideally like to excluding from applicability.
The rest of the elements I would like to check with the requirement of
"TestPset.TestProp" with value "Foo":

Specification:
    Applicability 1:
        Add all elements of
            IfcWall, IfcDoor, IfcWindow, IfcBuildingElementProxy

    Applicability 2:
        Remove from existing applicability all
            IfcBuildingElementProxy
                with "Attribute" "Name"
                    with regex value: "Pyramid.*"

    Requirements:
        with a specific "TestPset.TestProp"
           with value "Foo".

Currently I cannot seem to be able to exclude the 6 pyramids,
so it gets them flagged, which I would like to avoid. 🤔

@Moult
Copy link
Contributor

Moult commented Sep 4, 2023

I suspect this is a relatively simple solution. Right now xs:occurs is only requirements. If we add xs:occurs to applicability, then we can treat Required as "filter applicable by matching this criteria" (current behaviour) and Prohibited as "filter applicable by not matching this critera" (i.e. the proposed subtraction).

https://github.com/buildingSMART/IDS/blob/master/Documentation/specifications.md#required-and-optional-specifications-and-facets

@CBenghi CBenghi added this to the 1.1 milestone Apr 19, 2024
@atomczak
Copy link
Contributor

Closing and moving the discussion to the idea about exceptions:
"Complete IDS by adding two more sections and recursion" #323

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants