Upsertion with projection: { _id: 0 } appears broken

[toptobes]

When doing findOneAndReplace with projection: { _id: 0 }, resulting in an upsert, the following is returned:

{
  data: {},
  errors: [{ message: 'Cannot invoke "com.fasterxml.jackson.databind.JsonNode.isArray()" because "subtree" is null' }],
}

where the data is interestingly non-null and the error field being quite... "internal"

Command sent:

{
  findOneAndReplace: {
    filter: { 'address.city': 'nyc' },
    replacement: {
      username: 'jimr',
      human: true,
      age: 52,
      password: 'gasxaq==',
      address: [Object]
    },
    options: { returnDocument: 'before', upsert: true },
    projection: { _id: 0 }
  }
}

The only document in the collection at the time (causing an upsert)

{
  username: 'aaron',
  human: true,
  age: 47,
  password: null,
  address: {
    number: 86,
    street: 'monkey street',
    suburb: 'not null',
    city: 'big banana',
    is_office: false,
  },
}

This may or may not happen with other commands as well, haven't tested.

[tatu-at-datastax]

I think what’d be great would be actual JSON being sent: what is included looks more like Java/TypeScript Object and is missing possibly important information. For example, on request sent we have:

address: [Object]

which is missing JSON Object contents, and the document that exists seems to not have _id (which must exist for all Documents). This information would be needed to reproduce the problem.

[tatu-at-datastax]

@toptobes I created a test (#1026) that tries to do what you describe above (with the caveats that I may be missing information as per my earlier comment). Test passes, and I suspect that one of changes since your report has fixed the issue related to projection handling for this case.

If you could retry test with the latest version of Data API as deployed that would be great: if the issue still persists, I'd need some more information, otherwise it can be closed.

[toptobes]

These should reproduce it?

{
  "insertOne": {
    "document": {
      "_id": 1
    }
  }
}

{
  "findOneAndReplace": {
    "filter": {
      "_id": 2
    },
    "replacement": {},
    "options": {
      "returnDocument": "before",
      "upsert": true
    },
    "projection": {
      "*": 0
    }
  }
}

Projection can be either "*": 0 or "_id": 0

[tatu-at-datastax]

@toptobes Ok, hmmh. I used "_id": 0 projection in the test. I think the key is it needs to be anything but not the "default" projection, which is (or at least used to be) skipping part of processing.

[toptobes]

Hmm it seems to work fine when the projection is just { _id: 1 }, but everything else seems to break it. If I do projection: { john: 1, _id: 1 }, it returns the same error, when doing the following, even though no other field is even present:

await collection.insertOne({ _id: 1, john: '1' });

await collection.replaceOne(
  { _id: 2 },
  { john: '1' },
  { upsert: true, },
);

[tatu-at-datastax]

Definitely sounds like applying any projection other than the new "include all" ({"*":1}) on non-existing "before" document would trigger this.

But I am not yet able to reproduce this against code in "main". Still trying.

[tatu-at-datastax]

Was able to make test from #1026 fail against v1.0.5 tag; passes now against main.
So fix is included in 1.0.6, via another PR (most likely #1016 fixed it).
Will leave open until 1.0.6 release.