Issue with Delta import of Snomed Uk #699
-
Hello, We are having some issues with the import of a Delta Snomed Uk update. This is our structure:
Our steps are the following:
The problem is that, with archive SNOMEDCT2_28.0.0_20191001000001, the import doesn't produce any commit on the branch. There are no errors in the log, except for a few unrecognized snomed ids, and the import apparently succeeds, but the branch is just empty. We have tried with other archives and it works perfectly fine, but we need this one first. Any suggestion on what we could be doing wrong or anything we can check? Thanks, |
Beta Was this translation helpful? Give feedback.
Replies: 15 comments 15 replies
-
Could you please send us some relevant logs from Snow Owl to aid the investigation? Usually, the import produces no commit because of the following two reasons:
Thank you, |
Beta Was this translation helpful? Give feedback.
-
Hi Mark, SNOMEDCT2_28.0.0_20191001000001 an official UK Clinical RF2 (Delta only) archive. The current content is in branch MAIN/2018-07-31/SNOMEDCT-UK/2019-06-01 while the import attempt was in branch MAIN/2018-07-31/SNOMEDCT-UK/2019-10-01 By manually looking at the archive files, it looks like there are at least a few new concepts to import in that release, so I think it is unlikely that there is nothing to commit. But there were some error messages, so your second option may explain the issue. Please find below the logs of the import attempt.
Thanks, |
Beta Was this translation helpful? Give feedback.
-
That is the source of the issue then. If the data lives on the SNOMEDCT-UK branch, then feel free to import it onto that branch with the Hint: The Extension management documentation has very descriptive images and scenarios to help you understand Snow Owl's branching/versioning features and how an extension can be represented in Snow Owl. I hope this clarifies your question, let me know if you have any further. Cheers, |
Beta Was this translation helpful? Give feedback.
-
Hi @cmark, Alternatively, do you have any plan of exposing the import validation as a separate endpoint? Thanks, |
Beta Was this translation helpful? Give feedback.
-
Yes, I understand hacking the RF2 file is not the best solution when it comes to importing it without errors into Snow Owl.
Let us know if you have any further issues with RF2 imports. Thank you for your feedback and ideas, |
Beta Was this translation helpful? Give feedback.
-
Hi @cmark, I was able to make progress with the delta updates, but I'm now stuck with one specific archive. I am getting a lot of validation errors referring to missing IDs contained in the file der2_cRefset_CareRecordElementDelta_GB_20200513.txt. I wanted to check if you tried to use this archive and if you run into the same issues. Thanks, |
Beta Was this translation helpful? Give feedback.
-
I'm happy to hear you were able to make progress. Yes, UK RefSets often refer content from another extension, especially if the member is coming from the UK EDITION.
Yes, we are maintaining servers with all the UK data up to the most recent releases (both 2020-08-05 and 2020-10-28) and we have identified those issues as well and in order to import them, we had to fix them first in the RF2 archive. I hope this helps and as always feel free to reach us if you run into any issues. Cheers, |
Beta Was this translation helpful? Give feedback.
-
FYI: The latest version has an improved RF2 import API, where you will be able to run the import process in Let us know what you think. Cheers, |
Beta Was this translation helpful? Give feedback.
-
Thank you for letting me know Mark. |
Beta Was this translation helpful? Give feedback.
-
Hi @cmark, I have now a slightly different question, still related to the imports. Assume I have my Snomed UK in a branch like this MAIN/2018-07-31/SNOMED-UK. What would be the right approach here? Again, thanks for your help! :) |
Beta Was this translation helpful? Give feedback.
-
Good morning Mark, So, to recall, currently we have MAIN/2018-07-31/SNOMEDCT-UK/2019-06-01 and we start a series of delta updates from there. What I do:
Do you see something wrong in the above steps that would cause this to happen? I should mention, I also tried to merge the latest uk branch - MAIN/2018-07-31/SNOMEDCT-UK/2020-06-10 directly into MAIN/2019-07-31/SNOMEDCT-UK, but the result is still the same (i.e., 17k errors). Cheers, Cristina |
Beta Was this translation helpful? Give feedback.
-
Hi @cmark ,
Cheers, |
Beta Was this translation helpful? Give feedback.
-
Hi Mark, Thanks again, |
Beta Was this translation helpful? Give feedback.
-
Hi @cmark, At the point in which i merge with "source": "MAIN/2018-07-31/SNOMEDCT-UK", "target": "MAIN/2019-07-31/SNOMEDCT-UK" and "squash": false, I can now see UK concepts in both branches, but I cannot see Snomed International concepts. More precisely, assume I try to lookup international concept XYZ, I can see it in MAIN/2018-07-31/SNOMEDCT-UK, I can see it in MAIN/2019-07-31, but I cannot see it in MAIN/2019-07-31/SNOMEDCT-UK, where I get an error 500. The following attempts at imports fail again due to many references to international codes that cannot be found. The merge status is completed and not log indicates any failure. In this case the issue seems to be the child branch is not inheriting the parent branch concepts as expected. Any idea what may be going on? Cheers, Cristina |
Beta Was this translation helpful? Give feedback.
-
Hi @cmark, The issue with the merge seems to be resolved, but I still cannot do a clean import of the August 2020 Uk release. I now have MAIN/2018-07-31/SNOMEDCT-UK and MAIN/2019-07-31/SNOMEDCT-UK, which have exactly the same number of concepts, and have updated the dependency of SNOMEDCT-UK to the 2019 branch. I want to import the August Delta, dependent on the 2019 release, and the following happens:
Of course we could keep importing everything into the 2018 branch, but this would result in a wrong branching structure going forward. Do you have any other suggestions of what could be causing the issue? Cheers, |
Beta Was this translation helpful? Give feedback.
Hi @cristinacivili-work,
FYI: The latest version has an improved RF2 import API, where you will be able to run the import process in
dryRun
mode, where only the validation phase is going to be executed and the actual import will be skipped. The response object should also contain information about potential import file defects and aid you in locating and resolving them.Let us know what you think.
Cheers,
Mark