New snapshot config to validate uniqueness before merge

[graciegoheen]

          @kjstultz thanks again for creating this issue! Appreciate that you invested the time at Coalesce in NOLA to figure out how to reproduce this tricky situation.

Reading the discussion between you and @jtcohen6, it sounds like the feature discussed thus far would look like this:

• Add a new snapshot config parameter validate_uniqueness (default to false)
• When validate_uniqueness is true, execute a uniqueness test after creating the staging table and before running the merge

Although this proposed feature wouldn't fix a snapshot table that already has duplicates, it would prevent duplicates from being added in the first place! Feels like a win to me, especially with it being configurable for those that (somehow) know that their "unique" key is truly unique.

Originally posted by @dbeatty10 in #6089 (comment)

---

Notes:

• Would be slow in some cases, so want to make sure this is configurable
• But valuable to make sure snapshot is accurate