Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Validator Generates False-Positives #72

Closed
hhund opened this issue Jun 22, 2022 · 12 comments · Fixed by #73, #76, #78 or #88
Closed

Validator Generates False-Positives #72

hhund opened this issue Jun 22, 2022 · 12 comments · Fixed by #73, #76, #78 or #88
Assignees
@hhund
Labels
bug Something isn't working ready for release Issue is fixed and merged into develop, ready for next release
Milestone
v0.6.0

Comments

@hhund
Copy link
Member

hhund commented Jun 22, 2022

Effected process plugin version: 0.5.0

Due to an error in the StructureDefinition snapshot generator of HAPI 5.1.0, the validator produces false-positive errors. Effected StructureDefinitions validation rules from de.gecco|1.0.5 and its depended implementation guides are:

A fix already exists for the following rule in the 0.5.0 release of the process plugin:

Root cause of the false-positive validation errors are wrong generated "min = X" (for X != 0) rules in the StructureDefinition snapshot for slicing rules without a "min" definition where the slicing definition (incl. discriminator) is not part of the same profile but rather defined in the base profile.

The fix will include identifying bad profile rules and adding a min 0 where appropriate. Similar to the existing fix:

codex-processes-ap1/codex-process-data-transfer/src/main/java/de/netzwerk_universitaetsmedizin/codex/processes/data_transfer/validation/structure_definition/GeccoRadiologyProceduresCodingSliceMinFixer.java

Lines 15 to 31 in f5aaa4e

public StructureDefinition modify(StructureDefinition sd)
{
if ("https://www.netzwerk-universitaetsmedizin.de/fhir/StructureDefinition/radiology-procedures"
.equals(sd.getUrl()) && "1.0.5".equals(sd.getVersion()))
{
sd.getDifferential().getElement().stream().filter(
e -> "Procedure.code.coding".equals(e.getPath()) && e.hasMax() && e.hasSliceName() && !e.hasMin())
.forEach(e ->
{
logger.warn("Adding min=0 to rule with id {} in StructureDefinition {}|{}", e.getId(),
sd.getUrl(), sd.getVersion());
e.setMin(0);
});
}
return sd;
}

@hhund hhund added the bug Something isn't working label Jun 22, 2022
@hhund hhund added this to the v0.5.1 milestone Jun 22, 2022
@hhund hhund self-assigned this Jun 22, 2022
@hhund hhund mentioned this issue Jun 22, 2022
Merged
@hhund hhund linked a pull request Jun 22, 2022 that will close this issue
general profile slice min fix rule #73
Merged
@hhund hhund added the ready for release Issue is fixed and merged into develop, ready for next release label Jun 25, 2022
@hhund hhund mentioned this issue Jun 25, 2022
Merged
@hhund hhund closed this as completed in 2120b44 Jun 25, 2022
@eicherj
Copy link

eicherj commented Jun 28, 2022

When using codex-process-data-transfer-0.5.1.jar (via commandline), we receive the following validation error

{
	"severity": "error",
	"code": "processing",
	"diagnostics": "Could not confirm that the codes provided are in the value set https://www.netzwerk-universitaetsmedizin.de/fhir/ValueSet/rheumatological-immunological-diseases, and a code from this value set is required",
	"location": [
		"Condition.code.coding[1]",
		"Line 1, Col 1303"
	]
}

relating to the following code

{
	"coding": [
		{
			"system": "http://fhir.de/CodeSystem/dimdi/icd-10-gm",
			"version": "2021",
			"code": "D84.8"
		},
		{
			"system": "http://snomed.info/sct",
			"code": "396332003+414029004"
		}
	]
}

See the complete patient.txt, lines 1315-1324 and lines 1355-1360, respectively.

@hhund
Copy link
Member Author

hhund commented Jun 28, 2022

Reopening this issue due to additional false positive validation errors.

@hhund hhund reopened this Jun 28, 2022
@hhund hhund removed the ready for release Issue is fixed and merged into develop, ready for next release label Jun 28, 2022
@hhund hhund modified the milestones: v0.5.1, v0.5.2 Jun 28, 2022
@taneroksuz
Copy link

taneroksuz commented Jun 29, 2022 •
edited
Loading

We get follwing errors, when we validate our fhir bundle (only for testing purpose):

  1. bpe-app-1 | ERROR pool-2-thread-1 - ValidateData.logValidationDetails(164) | Validation error for https://fhir-store-test.uk-augsburg.science/fhir/Observation/7a23462b0b2743a3630177affdd9264a17dcc2ddb83b99ed4385a90d82def786/_history/1 location Observation: Profile https://www.netzwerk-universitaetsmedizin.de/fhir/StructureDefinition/sars-cov-2-ab-pnl-ser-pl-ia, Element 'Observation.identifier': minimum required = 1, but only found 0
  2. bpe-app-1 | ERROR pool-2-thread-1 - ValidateData.logValidationDetails(164) | Validation error for https://fhir-store-test.uk-augsburg.science/fhir/Observation/7a23462b0b2743a3630177affdd9264a17dcc2ddb83b99ed4385a90d82def786/_history/1 location Observation: Profile https://www.netzwerk-universitaetsmedizin.de/fhir/StructureDefinition/sars-cov-2-ab-pnl-ser-pl-ia, Element 'Observation.identifier[analyseBefundCode]': minimum required = 1, but only found 0
  3. bpe-app-1 | ERROR pool-2-thread-1 - ValidateData.logValidationDetails(164) | Validation error for https://fhir-store-test.uk-augsburg.science/fhir/Observation/e8b0bd67f9a5a2f689f7d2e1ea5c00d1b37214275a7f8300e71b9b0364447605/_history/1 location Observation.code: Profile https://www.netzwerk-universitaetsmedizin.de/fhir/StructureDefinition/oxygen-partial-pressure, Element 'Observation.code.coding': minimum required = 1, but only found 0
  4. bpe-app-1 | ERROR pool-2-thread-1 - ValidateData.logValidationDetails(164) | Validation error for https://fhir-store-test.uk-augsburg.science/fhir/Patient/1f06df569a74be3085a5b4dbfc31a3f09b32fc989b80ef4b39d39deefe272a5b/_history/3 location Patient.extension[0]: Profile https://www.netzwerk-universitaetsmedizin.de/fhir/StructureDefinition/age, Element 'Patient.extension[0].extension[dateTimeOfDocumentation]': minimum required = 1, but only found 0

Please download the fhir bundle with txt extension

@hhund
Copy link
Member Author

hhund commented Jun 29, 2022 •
edited
Loading

@taneroksuz Based on my understanding I am classifying the first two errors as false positive.

  • From a quick search the SarsCoV2AbPnlSerPlIA profile is currently the only profile missing from our ObservationIdentifierRemover class. As such it should be the last remaining profile suffering from the analyseBefundCode false positive error.

  • The third error about Observation.code.coding seems to be correct since your code does not match one of the 3 codes allowed for that location in the PaO2 profile:
    One of the following codes is expected:

    {
  "system": "http://loinc.org",
  "code": "11556-8"
}

{
  "system": "http://loinc.org",
  "code": "2703-7"
}

{
  "system": "http://loinc.org",
  "code": "2704-5"
}

    The code 11557-8 from your example, does not seem to exist within the LOINC catalog: Code Not Found

  • The fourth error is also an error on your end. Within the Age extension profile, the data-absent-reason extension is not allowed as a stand-in for the dateTimeOfDocumentation value. I may have misjudged this during the hackathon. My understanding now is, that the Data Absent Reason extension can only be used as a stand-in if it is explicitly allowed by the profile, e.g. in MII Laboruntersuchung as a stand-in for Observation.value.

@hhund
Copy link
Member Author

hhund commented Jul 3, 2022

In addition to the code 396332003+414029004 included in ValueSet RheumatologicalImmunologicalDiseases reported by @eicherj, the following SNOMED CT codes included in ValueSet VaccinesSNOMED will currently (release 0.5.1) result in false-positive validation errors:

836498007+836388000
871729003+836380007+601000221108+863911006+836374004+871871008
871729003+836380007+601000221108+863911006+836374004
871729003+836374004+863911006
863911006+836374004+601000221108+836381006
836380007+1031000221108
871729003+836388000+863911006
836498007+836382004
836398006+836380007
836380007+601000221108

@hhund hhund mentioned this issue Jul 3, 2022
Merged
@hhund hhund linked a pull request Jul 3, 2022 that will close this issue
Issues/72 validator generates false positives #78
Merged
@hhund hhund modified the milestones: v0.5.2, v0.6.0 Jul 3, 2022
@hhund hhund added the ready for release Issue is fixed and merged into develop, ready for next release label Jul 5, 2022
@eicherj
Copy link

eicherj commented Jul 7, 2022

We are getting the following validation error:

app_1 | ERROR pool-2-thread-1 - ValidateData.logValidationDetails(164) | Validation error for https://node.dev.codex.imedis.med.tum.de/fhir/Observation/4f32c9f55a5afff8504f275101ae6afafc209bf835773a9f04f7e2ab33a81ac0/_history/3 location Observation.value.ofType(Quantity).code: Value is '{beats}/min' but must be '/min'

However, the UCUM specification (https://ucum.org/ucum.html) states:

"§6 curly braces ■1 The full range of characters 33–126 can be used within a pair of curly braces (‘{’ and ‘}’). The material enclosed in curly braces is called annotation. ■2 Annotations do not contribute to the semantics of the unit but are meaningless by definition. Therefore, any fully conformant parser must discard all annotations. Parsers of limited conformace should not value annotations in comparison of units. ■3 Annotations do, however, signify the end of a unit symbol. ■4 An annotation without a leading symbol implies the default unit 1 (the unity). ■5 Curly braces must not be nested".

Is that a known issue?

@hhund
Copy link
Member Author

hhund commented Jul 11, 2022

Hi @eicherj,

the GECCO profile Respiratory Rate has a validation rule for Observation.valueQuantity.code defining a fixed value using the pattern: /min

Since Observation.valueQuantity.code is a primitive and the FHIR R4 standard states

When pattern[x] is used to constrain a primitive, it means that the value provided in the pattern[x] must match the instance value exactly.

the UCUM specification is ignored. Based on the current Respiratory Rate profile, the validation error you mentioned above is not a false-positive, but rather a result of the way the GECCO profile is defined.

The validator from simplifier.net seems to agree with this assessment.

@julsas Any thoughts on changing Respiratory Rate and other profiles? I think in order to allow annotations in units we would need to remove the fixedPattern rules and add bindings to specific ValueSets. These ValueSets would then include one or multiple entries from the http://unitsofmeasure.org CodeSystem.
Implementing a parser based code in ValueSet check should be possible, since the HAPI FHIR Validator we are using as part of the process validation step, already supports this against the full http://hl7.org/fhir/ValueSet/ucum-units ValueSet.

@julsas
Copy link
Member

julsas commented Jul 12, 2022

@hhund @eicherj {beats}/min is a valid UCUM unit, but GECCO uses the UCUM units from the FHIR Core spec which mandates to use /min as in http://hl7.org/fhir/R4/resprate.html. Thus, I wouldn't change the profiles.

@hhund hhund mentioned this issue Jul 12, 2022
Merged
@hhund hhund closed this as completed in #88 Jul 12, 2022
@taneroksuz
Copy link

  • Most of the validation errors are fixed. There is only one validation error left. However this error is not related to client
    side but server side:
  • The validation output is given as follows: 
    bpe-app-1  | ERROR Grizzly(2) - LogValidationError.logValidationDetails(120) | CRR validation error for https://fhir-store-test.uk-augsburg.science/fhir/Observation/9b22a43accacc42f46c8c69bfba896a547fa7c58f28336e98c97a1f3094af59b/_history/1: StructureDefinition https://www.medizininformatik-initiative.de/fhir/core/modul-labor/StructureDefinition/QuelleKlinischesBezugsdatum has no snapshot, and no snapshot generator is configured
  • This error refers to following lines in fhir resources: 
    <effectiveDateTime>
  <extension url="http://hl7.org/fhir/StructureDefinition/data-absent-reason">
    <valueCode value="unknown"/>
  </extension>
</effectiveDateTime>
  • This error can be fixed as wir replace the extension with following line: 
    "effectiveDateTime": "1972-01-01T01:00:00+01:00"

fhir-bundle.txt

@franzplri
Copy link

franzplri commented Aug 23, 2022 •
edited
Loading

We had an error when validating Blood-Gas-Panel resources through the DSF. The error always looked like this:

app_1 | Validation error for Observation: Profile https://www.netzwerk-universitaetsmedizin.de/fhir/StructureDefinition/blood-gas-panel, Element 'Observation.identifier': minimum required = 1, but only found 0

app_1 | Validation error for Observation: Profile https://www.netzwerk-universitaetsmedizin.de/fhir/StructureDefinition/blood-gas-panel, Element 'Observation.identifier[analyseBefundCode]': minimum required = 1, but only found 0

Solution:
DSF still had an old version of the blood-gas-panel structure definition stored as a snapshot.

  1. Exit BPE
  2. Delete file: /opt/bpe/cache/StructureDefinition/https_www.netzwerk-universitaetsmedizin.de_fhir_StructureDefinition_blood-gas-panel_1.0.5.json.gz
  3. Start BPE

The structure definition is reloaded and everything works fine now.

@julsas
Copy link
Member

julsas commented Aug 31, 2022

  • Most of the validation errors are fixed. There is only one validation error left. However this error is not related to client
    side but server side:

    • The validation output is given as follows:

      bpe-app-1  | ERROR Grizzly(2) - LogValidationError.logValidationDetails(120) | CRR validation error for https://fhir-store-test.uk-augsburg.science/fhir/Observation/9b22a43accacc42f46c8c69bfba896a547fa7c58f28336e98c97a1f3094af59b/_history/1: StructureDefinition https://www.medizininformatik-initiative.de/fhir/core/modul-labor/StructureDefinition/QuelleKlinischesBezugsdatum has no snapshot, and no snapshot generator is configured

    • This error refers to following lines in fhir resources:

      <effectiveDateTime>
  <extension url="http://hl7.org/fhir/StructureDefinition/data-absent-reason">
    <valueCode value="unknown"/>
  </extension>
</effectiveDateTime>

    • This error can be fixed as wir replace the extension with following line:

      "effectiveDateTime": "1972-01-01T01:00:00+01:00"

fhir-bundle.txt

This is still an issue? Sounds related to the snapshot generator problem. Snapshot for this extension can be obtained from here: https://simplifier.net/packages/de.medizininformatikinitiative.kerndatensatz.laborbefund/1.0.6/files/388075

@hhund to fix this, couldn't the framework install the StructureDefs with pre-generated snapshots instead of using HAPI's snapshot generator?

@hhund
Copy link
Member Author

hhund commented Aug 31, 2022

The false positive validation error about the QuelleKlinischesBezugsdatum extensions is/was generated by the FHIR Bridge component of the NUM-CODEX central research repository during insert of FHIR resources using the extension.

As of today (2022-08-31 17:30) both production and the pre-prod (test) deployments have been fixed via a special release of the FHIR Bridge. The issue is otherwise being tracked at ehrbase/fhir-bridge#581

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@hhund hhund

Labels
bug Something isn't working ready for release Issue is fixed and merged into develop, ready for next release
Projects
None yet
Milestone
v0.6.0
5 participants
@eicherj @hhund @julsas @taneroksuz @franzplri