Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Define if and when it is OK to inject axioms into an external ontology #1443

Open
cmungall opened this issue Feb 18, 2021 · 58 comments
Open

Define if and when it is OK to inject axioms into an external ontology #1443

cmungall opened this issue Feb 18, 2021 · 58 comments
Labels
external resource Issues related to interactions with external (non-Foundry) resources principles Issues related to Foundry principles

Comments

@cmungall
Copy link
Contributor

cmungall commented Feb 18, 2021 •
edited
Loading

Previously we have had issues when one ontology injected axioms into another

For example, previously OBI injected an axiom GO:molecular_function subClassOf BFO:function

This did not contravene any stated OBO rule or principle, so there was formally nothing wrong with it. But intuitively this kind of thing is undesirable, as it should be up to the external ontology to state their axioms. If other ontologies state axioms on another ontology's behalf we will have issues of conflicts (as what happened here).

Of course, OBI later lifted this axiom, but this serves as a good example.

We may choose to formulate a rule: an ontology O MUST not create any axioms about a class C in any non-O ontology.

Note that here, there is a precise definition of about in the OWLAPI. Intuitively and roughly it is where C is on the LHS of the axiom.

Here by create we mean to make an axiom de-novo. Of course, O may include axioms from an external ontology derived via robot extract, and merge these in to O (we have strategies for avoiding injection of stale axioms with base modules).

This is a good start, but is not sufficient.

First, it would forbid any disjointness axioms crossing ontologies, as these are by definition about classes in two ontologies. I think we want to have a way of allowing this, but to encourage both ontologies to 'sign off' on this.

Similarly, it would forbid equivalence axioms between two named classes in two ontologies. We generally wouldn't expect this in OBO as we strive for orthogonality, but there may be temporary situations (e.g. GO:cell = CL:cell, now resolved by GO obsoleting their cell)

There is also a more nuanced situation, where we have injected entailed axioms

E.g. if I say MyOnt:drug synthesis = MyOnt:synthesis and RO:has-output some CHEBI:drug, and RO:has-output has a range restriction of material entity, I am injecting via entailment an axioms CHEBI:drug subClassOf mat-entity, which clearly conflicts with CHEBI's role hierarchy

similarly, when PRO (cc @nataled) has axioms that reference the identifiers.org URI for an HGNC gene (e.g. has-gene-template), then we are entailing:

  • that URI has an interpretation that is an OWL class
  • the superclass of that class conforms to the range restriction (in this case PRO doesn't have a range constraint- yet)

obviously this could be problematic later, a different ontology may make different ontological commitments of HGNC that conflict

This problem is compounded by the fact HGNC is not providing any ontological commitments, we are translating a database to OWL. I touch on this in my slides:

https://docs.google.com/presentation/d/1zKKCfFdbWA5JiNDdRQuvmvDpcjvwvMXEGgmzgnSYL2w/edit#slide=id.p

As a start, I propose a principle for an ontology O and any class C for a non-O ontology:

  • O MUST NOT make any direct axioms about C
  • O MAY create axioms that reference C, provided:
    • any entailed axioms about C are in accord with both axioms entailed within C's containing ontology AND with the stated intentions of that ontology
  • maintainers of O MAY make separate bridge ontologies that contravene these (e.g. NC-NC-equivalence axioms) provided the bridge axioms are not merged or imported in O, and metadata clearly indicates that the bridge ontology may inject asserted or entailed axioms that do not necessarily correspond to the intentions of the external ontology

The "stated intentions" part could be documented according to some standard template. Where this involves an ontology converted from a database to OWL, we will need guidelines on how the OWL-naive upstream will understand and state conformance with ontological commitments
@nlharris nlharris added policy Issues and discussion related to OBO Foundry policies attn: Operations Committee Issues pertinent to broad Foundry activities, such as policies and guidelines principles Issues related to Foundry principles and removed policy Issues and discussion related to OBO Foundry policies labels Feb 18, 2021
@matentzn
Copy link
Contributor

matentzn commented Feb 19, 2021 •
edited
Loading

I think this is really great. I wholeheartedly endorse this! You mainly talk about logical axioms here, which is the important bit; if we can get the OBO committee to agree on this, then we can open a new issue, one step further, that actually encourages annotations to go where they belong as well. For example, we now have quite a few ontologies defining synonyms, definitions etc for term from non-O - it would be very good if these could end up in non-O as well. But yes, separate issue, this here is much more important.

I guess what the big question is how this relates to our base program.

maintainers of O MAY make separate bridge ontologies that contravene these (e.g. NC-NC-equivalence axioms) provided the bridge axioms are not merged or imported in O

One could argue that these "glue" axioms are there because the ontology developer wants their class graph to look a certain way for their users - which is undesirable, but I would say, minus logic, minus bfo, fair enough. So I can see how my.owl (the main release artefact) could be considered a sort of massaged version of the ontology made for "the user".

What is however truly important, and I think there can't be any reasonable way to contradict this, is that the base ontologies adhere to this proposal absolutely stringently. It is kind of messy on how to deal with this technically. Maybe an axiom annotation excludeFromBase.

@dosumis
Copy link
Contributor

dosumis commented Feb 19, 2021

Strongly agree with the aims here.

+100 to annotation at the axiom level for axioms that contravene principle 1 (O MUST NOT make any direct axioms about C). It is inevitable that we have some of these (I've been guilt of adding a few; shared agreement is hard & sometimes we need to build before we've achieved it). As well as allowing these axioms to be weeded out from base so they don't pollute import chains. they could be used to generate reports that can be the basis for discussion with developers of the ontologies being imported with the aim of having these axioms live in source ontologies.

I also agree that tagging of local axioms for at least some types of APs is important.

Still a bit confused about how this applies to non-ontology sources. e.g. gene identifiers. Where we have to use them, is it not useful to have some reasonable level of classification in SO?

@matentzn
Copy link
Contributor

Should be discusssed at the call today

@jamesaoverton
Copy link
Member

This is the OBI issue that @cmungall refers to at the top: obi-ontology/obi#963

@cmungall
Copy link
Contributor Author

Brief notes from call today:

  • we should include something to the effect that O can inject axioms about C so long as they are consistent with cob-to-external. We want to avoid O unilaterally making decisions about where C belongs. COB provides the central forum/decision recording place

@cmungall
Copy link
Contributor Author

cmungall commented Jun 7, 2021

I propose we add a dashboard check

For any given ontology O, if O references or reuses any part of O2 (i.e there is any logical axiom whose signature contains a class from O2), then we do a test:

  1. robot merge O and O2 into O3, along with cob and cob-to-external
  2. run robot reason over O3 to determine if new incoherencies are introduced (omit this step if O2 is not coherent, though this should be flagged some other way)
  3. run robot report over O3 to determine if other non-logical problems are introduced

Step 2 will detect if O injects logical axioms inconsistent with O2's intent

Step 3 will detect cases such as O over-riding labels or text definitions or any maxCardinality=1 annotation properties

Note that steps 2 or 3 will flag cases where O's imports are out of date w.r.t O2 (we deliberately use whole ontologies not basefiles). I think it's useful to know when one's imports are out of sync, but I think we need a way of separating this scenario from "intentional injection".

So I would propose that O can optionally include as metadata versionIRIs of all referenced ontologies, and these are used in step 1. If this is not indicated, then the latest version is always used (and if O get's lots of red marks from using stale ontologies and not providing indication, it is on their head)

@matentzn
Copy link
Contributor

matentzn commented Jun 7, 2021

I think this is a great idea in general, but it will seriously increase the overall runtime of the dashboard.. especially since the some if the most frequently reused ontologies are also the largest (pro, go, ncbitaxon, obi etc). Also it is something really hard to distinguish if a reuse was actually intentional (think of a range constraint one an RO relation pulled in). Is there any smart way we could do this in a cheaper way?

@mellybelly
Copy link
Contributor

Perhaps slightly orthogonal, but can we explore examples related to NCIt - we have the obo version and the NCIt-plus version where we are trying to create interoperability (e.g. obo-compliant axioms) with a well renowned resource rather than reinventing. This happens regularly enough that we should have a strategy for handling cases like these. Or rather, it should happen more than it does at present so that OBO doesn't reinvent authoritative sources.

@alanruttenberg
Copy link
Member

alanruttenberg commented Jun 15, 2021 via email

@dosumis
Copy link
Contributor

dosumis commented Jun 15, 2021 via email

@cmungall
Copy link
Contributor Author

I think @alanruttenberg's concerns are valid, especially about under-axiomatized ontologies. In fact I have developed an entire ontology (CHIRO) that does nothing except inject axioms into CHEBI. But I am hopefully clear that this is a bandaid and that CHIRO should not exist.

I like @dosumis proposal, I think it addresses Alan's concerns, and furthermore it provides an easy way for us in OBO to survey the landscape and ask: what axioms are being intentionally injected, and where? what ontologies frequently get injected into? what are the reasons? is the target ontology unwilling to accept axiom suggestions? or are they under-resourced? how can we help?

@alanruttenberg
Copy link
Member

alanruttenberg commented Jun 15, 2021 via email

@alanruttenberg
Copy link
Member

alanruttenberg commented Jun 15, 2021 via email

@matentzn
Copy link
Contributor

I don't understand the problem with @cmungall suggestions 1,2,3 - If I understand it correctly I support it 100%, as well as what David suggests. The essence is that we need some way to prevent downstream ontologies to change the meaning of a term in any way - @alanruttenberg your notion of "correctness" would be of course a desirable guiding principle but is a lot harder to test and maintain than simply flagging up axioms with a tag. Also, you put IMO too much trust in the abilities of people to "correctly" infer the intended semantics from the textual definitions. Right now all our pipelines rely on stripping out all disjointness axioms so we can reason with even 2 ontologies together - this needs to stop. So we need an easy way to remove such axioms if they are needed for classification or something. Or do you have a concrete idea on how else to solve this? I just don't see another, but happy to change my mind!

I do however agree that none of this needs to happen manually - its easy to build the annotation process into robot (I would do it).

@cmungall cmungall mentioned this issue Jun 16, 2021
Closed
@cmungall
Copy link
Contributor Author

Just an addendum to point out another hidden problem others may not be aware of

injected axioms are currently stripped in base modules. This is good if I want to avoid injected axioms and the attendant incoherencies. However, it is bad in that it masks the problem, and it makes it harder to automatically detect if multi-ontology incoherencies are arising from version superpositions or are issues with asserted axioms.

An example is here, combining foodon and obi, both of which inject. This is masked by using the obi base but this just masks the problem:

obi-ontology/obi#1375

@cmungall cmungall mentioned this issue Jun 17, 2021
Closed
@cmungall
Copy link
Contributor Author

Note that I am myself guilty of horrendous injections which virally propagate. This was a really hard one to debug:

EnvironmentOntology/envo#1187

it is quite hilariously recursive and ironic, an (unintentional) injection on the editor preferred label of editor preferred label, rendering something that is the opposite of an editor preferred label.

@cmungall cmungall mentioned this issue Jun 20, 2021
Closed
@matentzn
Copy link
Contributor

Action items:

  • Break this ticket down into small decisions and call a vote

@matentzn matentzn removed the attn: Operations Committee Issues pertinent to broad Foundry activities, such as policies and guidelines label Jul 13, 2021
@cmungall cmungall mentioned this issue Aug 2, 2021
Closed
@cmungall cmungall mentioned this issue Aug 25, 2021
Closed
@cmungall cmungall mentioned this issue Sep 29, 2021
Closed
@lschriml lschriml mentioned this issue Sep 29, 2021
Closed
2 tasks
@cmungall cmungall mentioned this issue Nov 3, 2021
Open
@alanruttenberg
Copy link
Member

The syntactic constraints form I'm thinking of is the sort used to define the OWL profiles, for example those for EL

@dosumis
Copy link
Contributor

dosumis commented Jan 19, 2022 •
edited
Loading

Very rough first attempt by example, focussing on logical axioms.

Axiom A1 in Ontology O1 is considered an injection into O2 if:

  1. A1 is a subClassOf axiom in which a class defined in O2 is the subject (the object may be a named class or anonymous class expression)
  2. A1 asserts equivalence to a class in O2 (this covers equivalence to named classes or anonymous class expressions)
  3. A1 asserts disjointness between a class in O2 and a class defined in any ontology other than O1.
  4. A1 asserts that an ObjectProperty defined in O2 is a subPropertyOf any other object property.
  5. A1 asserts a characteristic of an ObjectProperty defined in O2.
  6. A1 is a property chain in which all referenced properties are defined in ontologies other than O1. *
  7. A1 is a subClassOf GCI in which any class from O2 is on the left

For the avoidance of doubt - This list is not exhaustive.

* might need some more thought.

@dosumis
Copy link
Contributor

dosumis commented Jan 19, 2022

Would be great if we could formalise an agreed list in SPARQL and use this for auto-tagging.

@alanruttenberg
Copy link
Member

#2 doesn't seem to be a problem unless there are additional assertions on the class asserted equivalent. Isn't 2 (sans additional assertions) the common usage for application ontologies? And not all assertions against the equivalent class are necessarily problematic.

Still thinking about the other ones.

What would be great would be, for each of the cases, to document an actual problem that arose because of that kind of assertion.

@alanruttenberg
Copy link
Member

Ideally do not use the term "subject" or "object". E.g. (1) can be restated as something like "A1 is a SubClassOf axiom and the subClassExpression of A1 is a named class from O2 or a class expression that includes a named class from O2."

That way you are using the same language in the OWL 2 Spec. Well, almost. "named class" isn't used in the spec and "includes" paraphrases "has, in its signature,". Those can be given as definitions in the introduction.

That said, I think the "or a class expression" in (1) isn't clear, which is why I made up something more specific. I'm assuming you didn't mean a class expression that is literally in O2. But, I think what I wrote is too strong. For instance, suppose O1 is an antibody ontology. Suppose I have

SubClassOf(ObjectIntersectionOf(o2:protein ObjectSomeValuesFrom(has_part o3:epitopeThatBindsX)) o1:antibodyToX )

Paraphrased: Any protein with an <epitope that binds something> is an <antibody to that something>

I can't remember whether epitopes are considered parts or qualities but the answer doesn't change point of the example. The stuff between <> isn't intended as an expression, it's an invented label for named classes in o2 or o3.

@bpeters42
Copy link
Contributor

bpeters42 commented Jan 20, 2022 via email

@dosumis
Copy link
Contributor

dosumis commented Jan 20, 2022

@bpeters42 - strong agree. The issue here is with ontologies that serve as a reference and so are a source of modular imports to other reference ontologies and to application ontologies.

@wdduncan
Copy link
Member

@bpeters42 and @dosumis not trying to derail the current conversation (perhaps it can be moved to a separate issue) but could such agreement about axiom injection/removal provide a criterion for distinguishing between foundry and library ontologies? (I know this distinction is a long standing issue.)

@dosumis
Copy link
Contributor

dosumis commented Jan 20, 2022

but could such agreement about axiom injection/removal provide a criterion for distinguishing between foundry and library ontologies?

Definitely don't want to get into that here. As far as I'm concerned the only distinction at present is historical accident.

@bpeters42
Copy link
Contributor

bpeters42 commented Jan 20, 2022 via email

@alanruttenberg
Copy link
Member

@bpeters42 I think you have something there. That said, I think the distinction between an application ontology and reference ontology is a useful one, in principle, but that there's not a clean line separating the two. But still, it's a reasonable place to start thinking about this. We've thought of the difference between reference and application ontologies as structural, but there is a use component. I think there can be 'reference ontologies' in that they are adding new domain classes but that aren't necessarily intended to be used in conjunction with other ontologies generally, instead being adequate for a defined community.

@dosumis

(4) A1 asserts that an ObjectProperty defined in O2 is a subPropertyOf any other object property.

This is harmless if there aren't other object property assertions made of the super-property and, if there are, the logical consequences aren't downward. So adding a transitive super-property won't change the logic of the sub-property, but adding a domain/range, or saying it is reflexive will. Are those the kind of cases you are thinking about? How about trying to narrow this.

(3) A1 asserts disjointness between a class in O2 and a class defined in any ontology other than O1.

What if it is true that the classes are disjoint? What if proper reasoning in O1 depends on capturing this correctly? Under-axiomatization is a problem. Are all 'reference' ontologies going to be committed to adding all true disjoints? Even if it means having to import more terms that don't consider relevant?

Devils advocate: Maybe the distinction should be between ontologies that are strongly committed to axiomatization vs ones that are not? And then work on defining what a strong commitment to axiomatization means.

@alanruttenberg
Copy link
Member

@dosumis
(5) Makes sense. I can't think of obvious cases where it would make sense and if such a change is correct and if someone thinks the characteristic is correct then talking to the 02 devs should resolve it. The only case I can think of that varies is when 02 doesn't assert a characteristic because if added and used with 03 it would violate the global restrictions. In that case an ontology that isn't going to be used with O3 might want to gain some inference from asserting the characteristic.

Domain and range assertions and DisjointProperties assertions can also cause damage although I can imagine there cases where domain and range might legitimately be asserted.

The issue of global restriction violation when using combinations of ontology is messy. Use of an ObjectHasSelf comes to mind. Also an exception to my earlier comment about adding transitive characteristic to a super-property. It could interfere with adding cardinality restrictions, if they would make sense.

(6) Mixed about 6. Worth tagging for sure. One reason I can think of adding a chain of this sort might be because none of the devs of ontologies sourcing the properties in the chain think the chain axiom is their responsibility.

@StroemPhi StroemPhi mentioned this issue May 11, 2022
Closed
@cmungall
Copy link
Contributor Author

I am not sure how best to progress this issue

I think the underlying idea is clear and should be uncontroversial but it is hard to follow the chain of everyone's reasoning in this issue. Perhaps create a new issue with a new formulation of the proposal, close this, and refer back to it?

@cmungall cmungall mentioned this issue Jul 18, 2022
Open
@wdduncan
Copy link
Member

@cmungall I agree. There has been a lot of conversation about this. How about the following:

  • Reference/domain ontologies can't inject/delete axioms.
  • Project/application ontologies can inject/delete axioms.

@matentzn
Copy link
Contributor

Here is another example of why it is so important to get this done. Go to ontobee sparql endpoint and run:

prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> 
prefix owl: <http://www.w3.org/2002/07/owl#> 
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT distinct ?graph_uri 
WHERE { 
GRAPH ?graph_uri 
{ 
<http://purl.obolibrary.org/obo/UBERON_0001062> rdfs:subClassOf <http://purl.obolibrary.org/obo/BFO_0000040> } 
}

Since Uberon:anatomical entity has thousands of immaterial subclasses, every time we merge any of these ontologies (query results above) we break the entire ontology. Right now we are fighting to build anything resembling a coherent application ontology..

@alanruttenberg
Copy link
Member

This is objectively an error. There's no question that it's not cool to inject incorrect axioms and that they need to be outlawed. The question in my mind is how to set policy to avoid ruling out people adding correct axioms. Can this sort of thing be caught as part of QC? For instance, we can document an expectation that any two ontologies taken together to never result in an inconsistency. This can be checked by automated processes. I can imagine a background process that is always doing this sort of thing and logging problems. When an inconsistency is found, the explanation generator can also be automatically run so that on review it's easier to pinpoint what the problem is.

@matentzn
Copy link
Contributor

Unsatisfiable classes are only the most obviously wrong part of this issue, but yes, we have some plans to add a general consistency check to OBO ontologies including their dependencies and add that to the OBO dashboard.

@alanruttenberg
Copy link
Member

Is there an explicit policy statement currently that says ontologies in the library should (in principle) be able to be taken together and not generate inconsistencies or unsatisfiable classes?

@alanruttenberg
Copy link
Member

Is there one that even says each alone must not have unsatisfiable classes or unsats? I haven't sampled OBO ontologies recently, but it's not infrequent that I review ontologies that don't even pass this basic test.

@cmungall cmungall mentioned this issue Oct 5, 2022
Open
@cmungall cmungall mentioned this issue Mar 22, 2023
Closed
12 tasks
@StroemPhi StroemPhi mentioned this issue Jun 20, 2023
Open
@nataled nataled mentioned this issue Sep 26, 2023
Open
@cthoyt cthoyt mentioned this issue Oct 8, 2023
Closed
12 tasks
@cmungall
Copy link
Contributor Author

Is there one that even says each alone must not have unsatisfiable classes or unsats? I haven't sampled OBO ontologies recently, but it's not infrequent that I review ontologies that don't even pass this basic test.

We need to make this more explicit.

It's here under fp002 on the registration checklist:

http://obofoundry.org/docs/RegistrationChecklist

The ontology MUST be parseable in Protégé, ROBOT or OWLAPI and be logically consistent and coherent with a standard OWL reasoner such as ELK or HermiT (i.e. there should be no unintended unsatisfiable classes)

However, the page for this principle doesn't say anything about this:

You are correct in that in the old days it would be not uncommon to see many examples of incoherent ontologies. Thanks to the awesome work done by the OBO TWG and thanks to ODK and the dashboard that should now never happen for any active ontology.

Of course, it's very easy to get incoherency by combining two ontologies, which is the motivation for this issue.

@cmungall cmungall mentioned this issue Jan 31, 2024
Open
@deepakunni3 deepakunni3 mentioned this issue Oct 1, 2024
Open
14 tasks
@wdduncan wdduncan mentioned this issue Dec 21, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
external resource Issues related to interactions with external (non-Foundry) resources principles Issues related to Foundry principles
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
9 participants
@cmungall @dosumis @alanruttenberg @jamesaoverton @nlharris @mellybelly @wdduncan @matentzn @bpeters42