fix(checkpoint-mongodb): fix filter support, store state deltas, populate pendingWrites

[benjamincburns]

Breaking changes:

• Changes the format of stored documents in mongo, requiring existing databases to be migrated by calling the MongoDBSaver.migrate method.

Non-breaking changes:

• Adds filter logic to the list method to support the options.filter argument, fixes MongoDBSaver.*list yields nothing when options.filter is defined, and fixing it requires a breaking change #581
  • Adds basic document format version tagging and migration support
  • Migrates existing stored metadata to stored as BSON objects rather than binary data.
  • Includes a change to checkpoint-sqlite and checkpoint libraries to centralize an array that I'm using in multiple packages to sanitize filter inputs.
• Stores channel_values by version on change, respecting the newVersions argument to put, fixes MongoDbSaver ignores newVersions argument on put - not storing deltas #595
• Populates pendingWrites and pending_sends correctly, fixes MongoDBSaver.list returns CheckpointTuples without the pendingWrites field populated. #589
• Updates checkpoint-validation to no longer skip tests that exercise the above fixes

[nfcampos]

I think we might accept arbitrary metadata keys?

[benjamincburns]

I was just taking cues from PostgresSaver on this. Is there some mechanism for user code to modify the checkpoint metadata? It doesn't have an index signature, so I (perhaps incorrectly) assumed it was a complete type that originated internally. The typing cues are contradictory though, as options.filter is Record<string, unknown>. A type of Partial<CheckpointMetadata> would've been a clearer expression of intent there if it was indeed meant to be restricted.

Regardless, I added it in the RDBMS implementations because the list of filter keys is being directly concatenated into the query text (SQL injection potential), and as a result I also added it here for consistency.

Assuming arbitrary filter keys aren't desired, perhaps it would be better to throw on unknown keys. Silently dropping them could be confusing behavior, otherwise.

If they are desired, we should think carefully about how to sanitize the inputs for the RDBMS implementations.

[benjamincburns]

The ESLint failure is a false positive and it should be fixed by #633. It seems transient, and sometimes occurs in main.

[benjamincburns]

The ESLint failure is a false positive and it should be fixed by #633. It seems transient, and sometimes occurs in main.

Just rebased on current main, which should remove the ESLint violation.