Lack of DIF type in json-ld VPs causing dropped/ignored fields when normalized

[gmulhearn-anonyome]

Versions tested: 0.9.0 and 0.11.0

Missing DIF context/type in VP

When ACApy holders create a VP (as part of a DIF presentation exchange), the VP adds an extra field presentation_submission as the DIF spec says to do, however the @context and type for this field are not added to the VP document. For instance ACApy will create:

{
  "@context": [
    "https://www.w3.org/2018/credentials/v1"
  ],
  "type": [
    "VerifiablePresentation"
  ],
  "verifiableCredential": [...],
  "presentation_submission": {...},
  "proof": {...}
}

but it should be:

{
  "@context": [
    "https://www.w3.org/2018/credentials/v1",
    "https://identity.foundation/presentation-exchange/submission/v1"
  ],
  "type": [
    "VerifiablePresentation",
    "PresentationSubmission"
  ],
  "verifiableCredential": [...],
  "presentation_submission": {...},
  "proof": {...}
}

See DIF spec for document examples.

Dropped fields

Without this context, presentation_submission is effectively undefined, which can cause more strict document loaders to error out completely. I had a look at ACApy's urdna2015 normalization for these VPs, and the behaviour i found is that context-less fields are silently dropped from the VP. In this case, presentation_submission and all of it's inner fields are dropped when normalized (both when signing as the holder, and verifying as the verifier). Effectively the presentation_submission data is never signed by the ld-proof.

I added some logs around the _canonize function to check this:

    def _canonize(self, *, input, document_loader: DocumentLoaderMethod) -> str:
        """Canonize input document using URDNA2015 algorithm."""
        # application/n-quads format always returns str

        missing_properties = get_properties_without_context(input, document_loader)

        if len(missing_properties) > 0:
            LOGGER.warning("MISSING PROPERTIES FOUND!")
            raise LinkedDataProofException(
                f"{len(missing_properties)} attributes dropped. "
                f"Provide definitions in context to correct. {missing_properties}"
            )

        normalized = jsonld.normalize(
            input,
            {
                "algorithm": "URDNA2015",
                "format": "application/n-quads",
                "documentLoader": document_loader,
            },
        )
        LOGGER.warning("NORMALIZATION: INPUT: %s\n\nOUTPUT: %s\n\n", input, normalized)
        return normalized

Here's an example of the result:

INPUT: {'@context': ['https://www.w3.org/2018/credentials/v1'], 'type': ['VerifiablePresentation'], 'verifiableCredential': [{'@context': ['https://www.w3.org/2018/credentials/v1', 'https://w3id.org/citizenship/v1'], 'type': ['VerifiableCredential', 'PermanentResident'], 'issuer': 'did:sov:MNskjCH2i6UUDvzUCUBM5L', 'issuanceDate': '2023-11-24T06:59:04.116Z', 'credentialSubject': {'type': ['PermanentResident'], 'givenName': 'gn unce', 'familyName': 'fn unce', 'id': 'did:key:z6Mks34xU8Cv7gGwPbf86ZQgsa26AYbBr4QThyfwkH55XWtN'}, 'proof': {'type': 'Ed25519Signature2018', 'proofPurpose': 'assertionMethod', 'verificationMethod': 'did:sov:MNskjCH2i6UUDvzUCUBM5L#key-1', 'created': '2023-11-24T07:00:44.841931+00:00', 'jws': 'eyJhbGciOiAiRWREU0EiLCAiYjY0IjogZmFsc2UsICJjcml0IjogWyJiNjQiXX0..Ps_hsKTHU_1ZazuZaGsI-3RdWSyuBr6P4UL5xG401lt9qu9WLXb7T1k6p5OGKSgnXgE1D0e7NSqzFa5eYKHNDA'}}], 'presentation_submission': {'id': '9563dfba-1bbc-4fcc-93ba-10bdb2cfbcfb', 'descriptor_map': [{'id': '08c602a1-d4a7-4a33-ab6b-b917f1b7119d', 'format': 'ldp_vc', 'path': '$.verifiableCredential[0]'}]}}

OUTPUT: <did:key:z6Mks34xU8Cv7gGwPbf86ZQgsa26AYbBr4QThyfwkH55XWtN> <http://schema.org/familyName> "fn unce" _:c14n0 .
<did:key:z6Mks34xU8Cv7gGwPbf86ZQgsa26AYbBr4QThyfwkH55XWtN> <http://schema.org/givenName> "gn unce" _:c14n0 .
<did:key:z6Mks34xU8Cv7gGwPbf86ZQgsa26AYbBr4QThyfwkH55XWtN> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <https://w3id.org/citizenship#PermanentResident> _:c14n0 .
_:c14n1 <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <https://www.w3.org/2018/credentials#VerifiablePresentation> .
_:c14n1 <https://www.w3.org/2018/credentials#verifiableCredential> _:c14n0 .
_:c14n2 <http://purl.org/dc/terms/created> "2023-11-24T07:00:44.841931+00:00"^^<http://www.w3.org/2001/XMLSchema#dateTime> _:c14n3 .
_:c14n2 <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <https://w3id.org/security#Ed25519Signature2018> _:c14n3 .
_:c14n2 <https://w3id.org/security#jws> "eyJhbGciOiAiRWREU0EiLCAiYjY0IjogZmFsc2UsICJjcml0IjogWyJiNjQiXX0..Ps_hsKTHU_1ZazuZaGsI-3RdWSyuBr6P4UL5xG401lt9qu9WLXb7T1k6p5OGKSgnXgE1D0e7NSqzFa5eYKHNDA" _:c14n3 .
_:c14n2 <https://w3id.org/security#proofPurpose> <https://w3id.org/security#assertionMethod> _:c14n3 .
_:c14n2 <https://w3id.org/security#verificationMethod> <did:sov:MNskjCH2i6UUDvzUCUBM5L#key-1> _:c14n3 .
_:c14n4 <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <https://w3id.org/citizenship#PermanentResident> _:c14n0 .
_:c14n4 <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <https://www.w3.org/2018/credentials#VerifiableCredential> _:c14n0 .
_:c14n4 <https://w3id.org/security#proof> _:c14n3 _:c14n0 .
_:c14n4 <https://www.w3.org/2018/credentials#credentialSubject> <did:key:z6Mks34xU8Cv7gGwPbf86ZQgsa26AYbBr4QThyfwkH55XWtN> _:c14n0 .
_:c14n4 <https://www.w3.org/2018/credentials#issuanceDate> "2023-11-24T06:59:04.116Z"^^<http://www.w3.org/2001/XMLSchema#dateTime> _:c14n0 .
_:c14n4 <https://www.w3.org/2018/credentials#issuer> <did:sov:MNskjCH2i6UUDvzUCUBM5L> _:c14n0 .

the presentation_submission field is silently dropped from the normalized data (used for signing and verifying the ldproof).

I believe the get_properties_without_context is meant to safe guard this from happening, however the first line of code for get_properties_without_context is a get-out-of-jail-free-card for VPs:

    # FIXME: this doesn't work with nested @context structures...
    if "verifiableCredential" in document:
        return []

I'm not sure exactly how concerning the dropped fields are in this case; i see no reason why the presentation_submission needs signature integrity, it just seems like a convenience to include it in the VP document. Either way, i believe acapy definitely should be typing and signing it, and may need some caution around the normalization behaviour of dropping context-less fields

[swcurran]

@PatStLouis -- perhaps this is the source of your issue in #2693 ?

[PatStLouis]

This seems like a different issue, I don't think it is related