Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Snowstorm ECL return inactive concepts #616

Open
liquid36 opened this issue Aug 14, 2024 · 3 comments
Open

Snowstorm ECL return inactive concepts #616

liquid36 opened this issue Aug 14, 2024 · 3 comments
Assignees
Labels

Comments

@liquid36
Copy link
Contributor

Hi! I'm dealing with a posible issue. When I request snowstorm for an ECL, it return inactive concepts.

I tried it in my local machine and in the Snomed Browser.
I imported this version SnomedCT_InternationalRF2_PRODUCTION_20240301T120000Z.zip into MAIN.

ECL: ^446608001

curl -X 'GET' \
  'http://localhost:8080/MAIN/concepts?activeFilter=true&termActive=true&language=es&ecl=%5E446608001&includeLeafFlag=false&form=inferred&returnIdOnly=true&offset=0&limit=10000' \
  -H 'accept: application/json' \
  -H 'Accept-Language: en'
https://browser.ihtsdotools.org/snowstorm/snomed-ct/MAIN/2024-03-01/concepts?activeFilter=true&termActive=true&language=es&ecl=%5E446608001&includeLeafFlag=false&form=inferred&returnIdOnly=true&offset=0&limit=10000
{
  "items": [ 
   ....
    "243955001",
    "312539002",
    "243079002",
    "77208007",
    "6541001",
    "162119005",
    "314926009",
    "263390006",
    "243931006",
    "368597006",
    "362291003",
    "331610001",
    "20008000",
    "69515008",
    "272689002",
    "367629006",
    "149885003",
    "58477004",
    "36064001",
    "182051002",
    "243946004",
    "174363005",
    "699982007",
    "245793003",
    "280244007",
  ],
  "total": 23600,
  "limit": 10000,
  "offset": 0,
  "searchAfter": "WzI3ODg5MTAwM10=",
  "searchAfterArray": [
    278891003
  ]
}

In the response, i found the concept 362291003 which is inactive in the current branch.

https://browser.ihtsdotools.org/snowstorm/snomed-ct/browser/MAIN/2024-03-01/concepts/362291003?descendantCountForm=inferred

{
  "conceptId" : "362291003",
  "descendantCount" : 0,
  "fsn" : {
    "term" : "Entire prosencephalon (body structure)",
    "lang" : "en"
  },
  "pt" : {
    "term" : "Entire prosencephalon",
    "lang" : "en"
  },
  "active" : false,
  "effectiveTime" : "20240101",
  "released" : true,
  "releasedEffectiveTime" : 20240101,
  "inactivationIndicator" : "AMBIGUOUS",
  "associationTargets" : {
    "POSSIBLY_EQUIVALENT_TO" : [ "263353005", "730443005" ]
  },
  "moduleId" : "900000000000207008",
  "definitionStatus" : "PRIMITIVE",
  "descriptions" : [ {
    "active" : true,
    "moduleId" : "900000000000207008",
    "released" : true,
    "releasedEffectiveTime" : 20170731,
    "descriptionId" : "748214019",
    "term" : "Entire prosencephalon (body structure)",
    "conceptId" : "362291003",
    "typeId" : "900000000000003001",
    "acceptabilityMap" : {
      "900000000000509007" : "PREFERRED",
      "900000000000508004" : "PREFERRED"
    },
    "type" : "FSN",
    "lang" : "en",
    "inactivationIndicator" : "CONCEPT_NON_CURRENT",
    "caseSignificance" : "CASE_INSENSITIVE",
    "effectiveTime" : "20170731"
  }, {
    "active" : true,
    "moduleId" : "900000000000207008",
    "released" : true,
    "releasedEffectiveTime" : 20170731,
    "descriptionId" : "479899012",
    "term" : "Entire prosencephalon",
    "conceptId" : "362291003",
    "typeId" : "900000000000013009",
    "acceptabilityMap" : {
      "900000000000509007" : "PREFERRED",
      "900000000000508004" : "PREFERRED"
    },
    "type" : "SYNONYM",
    "lang" : "en",
    "inactivationIndicator" : "CONCEPT_NON_CURRENT",
    "caseSignificance" : "CASE_INSENSITIVE",
    "effectiveTime" : "20170731"
  } ],
  "annotations" : [ ],
  "classAxioms" : [ {
    "axiomId" : "be80f815-c5a0-4db9-b1fe-9c2e215d7772",
    "moduleId" : "900000000000207008",
    "active" : false,
    "released" : true,
    "definitionStatusId" : "900000000000074008",
    "relationships" : [ {
      "active" : true,
      "moduleId" : "900000000000207008",
      "released" : false,
      "sourceId" : "362291003",
      "destinationId" : "81861006",
      "typeId" : "116680003",
      "type" : {
        "conceptId" : "116680003",
        "active" : true,
        "definitionStatus" : "PRIMITIVE",
        "moduleId" : "900000000000012004",
        "fsn" : {
          "term" : "Is a (attribute)",
          "lang" : "en"
        },
        "pt" : {
          "term" : "Is a",
          "lang" : "en"
        },
        "id" : "116680003"
      },
      "target" : {
        "conceptId" : "81861006",
        "active" : false,
        "definitionStatus" : "PRIMITIVE",
        "moduleId" : "900000000000207008",
        "fsn" : {
          "term" : "Prosencephalon structure (body structure)",
          "lang" : "en"
        },
        "pt" : {
          "term" : "Prosencephalon structure",
          "lang" : "en"
        },
        "id" : "81861006"
      },
      "groupId" : 0,
      "modifier" : "EXISTENTIAL",
      "characteristicType" : "STATED_RELATIONSHIP"
    } ],
    "id" : "be80f815-c5a0-4db9-b1fe-9c2e215d7772",
    "definitionStatus" : "PRIMITIVE",
    "effectiveTime" : 20240101
  } ],
  "gciAxioms" : [ ],
  "relationships" : [ {
    "active" : false,
    "moduleId" : "900000000000207008",
    "released" : true,
    "releasedEffectiveTime" : 20190731,
    "relationshipId" : "3993335023",
    "sourceId" : "362291003",
    "destinationId" : "81861006",
    "typeId" : "116680003",
    "type" : {
      "conceptId" : "116680003",
      "active" : true,
      "definitionStatus" : "PRIMITIVE",
      "moduleId" : "900000000000012004",
      "fsn" : {
        "term" : "Is a (attribute)",
        "lang" : "en"
      },
      "pt" : {
        "term" : "Is a",
        "lang" : "en"
      },
      "id" : "116680003"
    },
    "target" : {
      "conceptId" : "81861006",
      "active" : false,
      "definitionStatus" : "PRIMITIVE",
      "moduleId" : "900000000000207008",
      "fsn" : {
        "term" : "Prosencephalon structure (body structure)",
        "lang" : "en"
      },
      "pt" : {
        "term" : "Prosencephalon structure",
        "lang" : "en"
      },
      "id" : "81861006"
    },
    "groupId" : 0,
    "modifier" : "EXISTENTIAL",
    "characteristicType" : "STATED_RELATIONSHIP",
    "effectiveTime" : "20190731",
    "id" : "3993335023"
  }, {
    "active" : false,
    "moduleId" : "900000000000207008",
    "released" : true,
    "releasedEffectiveTime" : 20240101,
    "relationshipId" : "179597027",
    "sourceId" : "362291003",
    "destinationId" : "81861006",
    "typeId" : "116680003",
    "type" : {
      "conceptId" : "116680003",
      "active" : true,
      "definitionStatus" : "PRIMITIVE",
      "moduleId" : "900000000000012004",
      "fsn" : {
        "term" : "Is a (attribute)",
        "lang" : "en"
      },
      "pt" : {
        "term" : "Is a",
        "lang" : "en"
      },
      "id" : "116680003"
    },
    "target" : {
      "conceptId" : "81861006",
      "active" : false,
      "definitionStatus" : "PRIMITIVE",
      "moduleId" : "900000000000207008",
      "fsn" : {
        "term" : "Prosencephalon structure (body structure)",
        "lang" : "en"
      },
      "pt" : {
        "term" : "Prosencephalon structure",
        "lang" : "en"
      },
      "id" : "81861006"
    },
    "groupId" : 0,
    "modifier" : "EXISTENTIAL",
    "characteristicType" : "INFERRED_RELATIONSHIP",
    "effectiveTime" : "20240101",
    "id" : "179597027"
  }, {
    "active" : false,
    "moduleId" : "900000000000207008",
    "released" : true,
    "releasedEffectiveTime" : 20190131,
    "relationshipId" : "721005025",
    "sourceId" : "362291003",
    "destinationId" : "258335003",
    "typeId" : "123005000",
    "type" : {
      "conceptId" : "123005000",
      "active" : false,
      "definitionStatus" : "PRIMITIVE",
      "moduleId" : "900000000000012004",
      "fsn" : {
        "term" : "Part of (attribute)",
        "lang" : "en"
      },
      "pt" : {
        "term" : "Part of",
        "lang" : "en"
      },
      "id" : "123005000"
    },
    "target" : {
      "conceptId" : "258335003",
      "active" : true,
      "definitionStatus" : "PRIMITIVE",
      "moduleId" : "900000000000207008",
      "fsn" : {
        "term" : "Entire brain (body structure)",
        "lang" : "en"
      },
      "pt" : {
        "term" : "Entire brain",
        "lang" : "en"
      },
      "id" : "258335003"
    },
    "groupId" : 0,
    "modifier" : "EXISTENTIAL",
    "characteristicType" : "ADDITIONAL_RELATIONSHIP",
    "effectiveTime" : "20190131",
    "id" : "721005025"
  }, {
    "active" : false,
    "moduleId" : "900000000000207008",
    "released" : true,
    "releasedEffectiveTime" : 20190131,
    "relationshipId" : "721004026",
    "sourceId" : "362291003",
    "destinationId" : "57991002",
    "typeId" : "123005000",
    "type" : {
      "conceptId" : "123005000",
      "active" : false,
      "definitionStatus" : "PRIMITIVE",
      "moduleId" : "900000000000012004",
      "fsn" : {
        "term" : "Part of (attribute)",
        "lang" : "en"
      },
      "pt" : {
        "term" : "Part of",
        "lang" : "en"
      },
      "id" : "123005000"
    },
    "target" : {
      "conceptId" : "57991002",
      "active" : true,
      "definitionStatus" : "PRIMITIVE",
      "moduleId" : "900000000000207008",
      "fsn" : {
        "term" : "Entire embryo (body structure)",
        "lang" : "en"
      },
      "pt" : {
        "term" : "Entire embryo",
        "lang" : "en"
      },
      "id" : "57991002"
    },
    "groupId" : 0,
    "modifier" : "EXISTENTIAL",
    "characteristicType" : "ADDITIONAL_RELATIONSHIP",
    "effectiveTime" : "20190131",
    "id" : "721004026"
  }, {
    "active" : false,
    "moduleId" : "900000000000207008",
    "released" : true,
    "releasedEffectiveTime" : 20190131,
    "relationshipId" : "721006029",
    "sourceId" : "362291003",
    "destinationId" : "83418008",
    "typeId" : "123005000",
    "type" : {
      "conceptId" : "123005000",
      "active" : false,
      "definitionStatus" : "PRIMITIVE",
      "moduleId" : "900000000000012004",
      "fsn" : {
        "term" : "Part of (attribute)",
        "lang" : "en"
      },
      "pt" : {
        "term" : "Part of",
        "lang" : "en"
      },
      "id" : "123005000"
    },
    "target" : {
      "conceptId" : "83418008",
      "active" : true,
      "definitionStatus" : "PRIMITIVE",
      "moduleId" : "900000000000207008",
      "fsn" : {
        "term" : "Entire fetus (body structure)",
        "lang" : "en"
      },
      "pt" : {
        "term" : "Entire fetus",
        "lang" : "en"
      },
      "id" : "83418008"
    },
    "groupId" : 0,
    "modifier" : "EXISTENTIAL",
    "characteristicType" : "ADDITIONAL_RELATIONSHIP",
    "effectiveTime" : "20190131",
    "id" : "721006029"
  } ],
  "alternateIdentifiers" : [ ],
  "validationResults" : [ ]
}

What am i missing?

@kaicode kaicode self-assigned this Aug 14, 2024
@kaicode
Copy link
Member

kaicode commented Aug 14, 2024

The ECL ^ "Member of" operator selects the current members of a given reference set. Regardless of if the concept itself is active or not. However it will not include concepts referenced in inactive refset member rows.

This is the active member of the SNOMED CT to ICD-10 reference set that references the inactive concept in your example:
https://browser.ihtsdotools.org/snowstorm/snomed-ct/MAIN/2024-03-01/members?referenceSet=446608001&referencedComponentId=362291003

The Member of operator results in different behaviour to the hierarchy based ECL operators. When using hierarchy operators only active concepts are returned because inactive concepts have no active relationships, and are therefore not part of the hierarchy. However an inactive concept can still be a member of a reference set.

The inactive concepts within a reference set can be found using the member operator and the concept filter. For example:

^ 446608001 |SNOMED CT to ICD-O simple map reference set|  {{ C active = false }}

This SNOMED CT to ICD-O map contains 82 inactive concepts in the latest (August 2024) International Edition.

There are some reference sets that only contain inactive concepts, for example the inactivation indicator and historical association reference sets.

I hope that helps.

@liquid36
Copy link
Contributor Author

I tried the concept filter in the ECL and it worked fine. What is the purpose of activeFilter param?

Thanks.

@kaicode
Copy link
Member

kaicode commented Aug 15, 2024

You have a good point. The activeFilter is supposed to work, even when running ECL.

It looks like there is a bug: the active filtering is not being applied when using the returnIdOnly parameter.

These two requests, with the same ecl and activeFilter give a different results count:

The returnIdOnly parameter is intended for more advanced users who need better performance. Applying the ECL concept filter {{ C active = false }} is a good workaround.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants