feat: use a local static cache for commonly used contexts

[chumbert]

Related to this issue

• Package a list of commonly used contexts
• Replace the context loader from pyld with StaticCacheJsonLdDownloader
• StaticCacheJsonLdDownloader resolves contexts from local package in priority, then delegates to a downloader
• As of now, JsonLdDocumentDownloader and JsonLdDocumentParser are simply lifted from pyld in a way that decouples downloading and parsing JSON-LD documents.
• Tests are mostly drafts, waiting for feedback before investing time there.

[swcurran]

Could you explain the mechanics here? I’m trying to see the connection between the locally stored file and the URL.

How do we detect changes in the static files — at least at build time? I’m assuming that is a concern.

Presumably, any tests would fail the build time tests, which is a pain, but better than failing in a production environment :-).

This is good stuff — thanks.

[chumbert]

Could you explain the mechanics here? I’m trying to see the connection between the locally stored file and the URL.

Of course, sorry for the sparse PR description.

In essence, the new loader that replaces the pyld load checks contexts URLs against a mapping from URL to already loaded documents:

• StaticCacheJsonLdDownloader holds the mapping for known contexts.
• Loading documents from filesystem happens at startup when the conductor binds the DocumentLoader instance.
• This loading is implemented within the StaticCacheJsonLdDownloader constructor using the _load_jsonld_file function.
• Static contexts are bundled within aca-py python package and are stored in the project filestructure (you can see them in aries_cloudagent/vc/ld_proofs/resources.
• When aca-py tries to resolve a context, an URL present in the cache results in an immediate response, others go through the code that existed previously

How do we detect changes in the static files — at least at build time? I’m assuming that is a concern.

We don't and this is the pain of having files statically in the build. We'd need to keep contexts up-to-date. I'm not sure this is a big concern given that contexts are supposed to be versioned rather than changed as far as I understand. The medium term solution for aca-py would be to have the document loader be able to pick contexts from some configuration so that the burden of maintaining this list of known contexts is not solely on aca-py maintainers and contributors.

Presumably, any tests would fail the build time tests, which is a pain, but better than failing in a production environment :-).

As the files are present within the projects code, build time is unchanged, the only effect is that the package is marginally larger. I might be missing your point here.

[swcurran]

Thanks for the overview. Let us know when you are happy with this one for a review/merge.

Perhaps the “maintenance” question is a GHA that runs either only when triggered or perhaps on a cron job (once a month?) that checks the pre-loaded contexts against the static files and reports an error on a mismatch. It would need a good error message since it will happen so rarely :-).

[dbluhm]

I gave this an in depth look, made some very minor adjustments as recommended by pyright, and I'm happy with this. There's opportunity for more sophisticated stuff in the future but I think this gets us more than 90% of the way there.

I wish pyld was a little more... idiomatic python but I don't want to write anything to replace it lol. It does some things that are a little odd, especially next to other code in ACA-Py, but it's not too bad.

[dbluhm]

@swcurran for the sake of transparency, I'll let you do the honors of merging this PR since I made some small contributions to it.

[swcurran]

I did the merge in the wrong order, so this needs an update to main before it can be merged. Sorry about that.

[sonarcloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
No Duplication information

[dbluhm]

I did the merge in the wrong order, so this needs an update to main before it can be merged. Sorry about that.

I'll consider this an indication of intent to merge and will go ahead and push the button now that this branch is up to date and tests are passing!