-mongocOnly against a mongodb version 6 does not work properly

[FR-ADDIX]

We tried to run a MongoDB v6 with the orion-ld:1.2.0-PRE-1247.
Unfortunately this does not work very well.
What we noticed immediately, the subscriptions don't seem to survive the cache.
Existing subscriptions are no longer reached and new ones only seem to exist for a few seconds.

Our settings for orion-ld are:

• args:
  -dbhost mongodb-headless.fiware.svc.cluster.local:27017
  -logLevel DEBUG
  -ctxTimeout "10000"
  -experimental
  -mongocOnly

[kzangeli]

ok, I haven't tried this myself yet.
Waiting for mongodb to get official support for Ubuntu 22.04.

Do you have something interesting in the brokers logfile?

[FR-ADDIX]

We have not seen any anomalies in the logs.
Unfortunately, we did not notice the missing subscriptions immediately and only noticed it the next day when we had missing historical data.

[kzangeli]

To fix the problem, I need to be able to reproduce it.
So, can you give me a hint as to what you did, so I have somewhere to start, at least?

[FR-ADDIX]

We have a Kubernetes cluster (version 1.25).
In it we have a mongodb v6 running as a replica set.
As orion-ld we have orion-ld:1.2.0-PRE-1247 running.
The same constellation with mongodb5 and the orion-ld:1.1.0 or orion-ld:1.2.0 without -experimental and -mongocOnly did not give this error pattern.

[kzangeli]

By any chance, are you using the HTTP header "Fiware-Service" (NGSIv2) instead of "NGSILD-Tenant"?

[kzangeli]

Ah, one more thing. Multi-tenancy is OFF by default (I really need to change this).
If the broker isn't started with -multiservice, tenants aren't used, everything goes to the "default tenant".

So, please check that too

[FR-ADDIX]

No we use "NGSILD-Tenant".
-multiservice I don't think we use, is this new or has it been around for a while?
Is it possible that the subscriptions we are looking for just end up in the wrong tenant?
I will check this now.

[FR-ADDIX]

-multiservice was not set for us, but when we set it it returns an error:
multiple use of option '-multiservice'

[kzangeli]

Ah, ok, so it's there "by default" for the container (somehow).
Good. That's not the problem then.
And, you use NGSILD-Tenant ...
That's all I have for now.

Would you like us to look at this together?
I'm not available until Monday after 15.00

[FR-ADDIX]

I just see in MongoDB the subcribtion is created under a collection csubs.
The orion-ld doesn't seem to see it after it is out of its cache.
We see the subcribtion for about 1 minute then it is gone for the orion.

`{
"_id" : "urn:ngsi-ld:subscription:2aec77ea-a317-11ed-918f-1af96903837f",
"description" : "QL Vehicle",
"name" : "Notify QL Vehicle Data",
"entities" : [
{
"type" : "https://uri.etsi.org/ngsi-ld/default-context/Vehicle",
"id" : ".*",
"isPattern" : "true",
"isTypePattern" : false
}
],
"conditions" : [
"https://uri.etsi.org/ngsi-ld/default-context/dateObserved"
],
"status" : "active",
"createdAt" : 1675355541.8067973,
"modifiedAt" : 1675355541.8067973,
"throttling" : NumberInt(0),
"expression" : {
"geometry" : "",
"coords" : "",
"georel" : "",
"geoproperty" : "",
"q" : "",
"mq" : ""
},
"format" : "normalized",
"reference" : "http://quantumleap-quantumleap.fiware.svc:8668/v2/notify",
"mimeType" : "application/json",
"headers" : {
"Fiware-Service" : "captn"
},
"attrs" : [

],
"custom" : false,
"servicePath" : "/#",
"blacklist" : false,
"ldContext" : "https://uri.etsi.org/ngsi-ld/v1/ngsi-ld-core-context.jsonld",
"count" : NumberLong(202),
"lastNotification" : 1675355589.0492446,
"lastSuccess" : 1675355589.0492446

}`

[kzangeli]

I was able to reproduce the error and I've fixed the problem.
Will take me a few hours though to merge the PR.
Hopefully before I go to sleep, so you can test again tomorrow morning!

[kzangeli]

Gonna take a little longer than I thought.
The fix to this problem gave me 5 errors in other functests, so, I will have to look it over more carefully.
Hope to have it solved before Monday morning.

[kzangeli]

Modified the fix and all tests are now working, see PR #1319 .

Please test and if all is good, proceed and close this issue

[FR-ADDIX]

Hm, ok but we need a docker in https://hub.docker.com/ to test this in our cluster.

[kzangeli]

Sure: https://hub.docker.com/r/fiware/orion-ld/tags

[FR-ADDIX]

Ah, ok latest

[kzangeli]

Well, use "the latest", but, avoid "latest" literally ... :)

[FR-ADDIX]

ok, the subscriptions are back.

[kzangeli]

That's good.
It's weird, in the email I see info on a separate problem, but not here on github ...

Anyway, that problem you found about duplicated entity id is highly interesting.
Would you be so kind and open a new issue about that problem and I will dive right into it.

To be able to reproduce I'd need you to describe as precise as you can what you do to encounter the problem.

This issue you can now close, as it is/was about the subscription problem when restarting the broker

[FR-ADDIX]

The duplicate entity ID was apparently, a defective optic on our switch, sorry.

[kzangeli]

ah, ok. Fantastic! :)

Thank you very much for reporting!

You found an important bug that I'm very happy to have removed before next release!!!