Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(NODE-3740): Implement root and top level key utf-8 validation settings for BSON #472

Merged
merged 13 commits into from
Nov 19, 2021

Conversation

gjchong25
Copy link
Contributor

@gjchong25 gjchong25 commented Nov 12, 2021

Description

Adds a new validation option in DeserializeOptions that allows all or a specific subset of top-level keys in a document to be excluded from strict utf-8 validation and instead applies the replacement character behavior. Tests are added to make sure that both global and specific-key subsets can have utf8 validation enabled and disabled.

What is the motivation for this change?

Default behavior is to have strict utf-8 validation (an error will be thrown if invalid utf8 is detected). This change gives users the ability to opt into character replacement behavior instead of strict utf-8 validation.

Double check the following

  • Ran npm run lint script
  • Self-review completed using the steps outlined here
  • PR title follows the correct format: <type>(NODE-xxxx)<!>: <description>
  • Changes are covered by tests
  • New TODOs have a related JIRA ticket

@gjchong25 gjchong25 changed the title feat(NODE-3740) Implement root and top level key utf-8 validation settings for BSON feat(NODE-3740): Implement root and top level key utf-8 validation settings for BSON Nov 12, 2021
Copy link
Contributor

@nbbeeken nbbeeken left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some initial feedback!

Side note: can you run npm install and push the package-lock changes, looks like its not sync-ed up with main from the last release.

src/parser/deserializer.ts Outdated Show resolved Hide resolved
src/parser/deserializer.ts Outdated Show resolved Hide resolved
src/parser/deserializer.ts Outdated Show resolved Hide resolved
src/parser/deserializer.ts Outdated Show resolved Hide resolved
src/parser/deserializer.ts Outdated Show resolved Hide resolved
test/node/utf8_tests.js Outdated Show resolved Hide resolved
test/node/tools/utils.js Show resolved Hide resolved
src/parser/deserializer.ts Outdated Show resolved Hide resolved
src/parser/deserializer.ts Outdated Show resolved Hide resolved
src/parser/deserializer.ts Outdated Show resolved Hide resolved
Copy link
Contributor

@nbbeeken nbbeeken left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

whoops accidentally cleared my review state

@nbbeeken
Copy link
Contributor

I'm going to try and investigate the strange rollup issue in CI tomorrow, but we can manually verify this works as expected in the browser if it comes to it.

@nbbeeken nbbeeken added the Team Review Needs review from team label Nov 17, 2021
@nbbeeken nbbeeken requested a review from durran November 17, 2021 22:44
Copy link
Contributor

@dariakp dariakp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still working on reviewing the rest of the code, but wanted to capture this small suggestion

src/parser/deserializer.ts Outdated Show resolved Hide resolved
src/parser/deserializer.ts Show resolved Hide resolved
Copy link
Contributor

@dariakp dariakp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Finished reviewing the code, still need to double check the tests

src/parser/deserializer.ts Outdated Show resolved Hide resolved
src/parser/deserializer.ts Outdated Show resolved Hide resolved
src/parser/deserializer.ts Outdated Show resolved Hide resolved
src/parser/deserializer.ts Outdated Show resolved Hide resolved
src/parser/deserializer.ts Outdated Show resolved Hide resolved
src/parser/deserializer.ts Outdated Show resolved Hide resolved
src/parser/deserializer.ts Outdated Show resolved Hide resolved
src/parser/deserializer.ts Outdated Show resolved Hide resolved
test/node/utf8_tests.js Show resolved Hide resolved
Copy link
Contributor

@dariakp dariakp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a couple of things left to clean up

src/parser/deserializer.ts Outdated Show resolved Hide resolved
src/parser/deserializer.ts Outdated Show resolved Hide resolved
src/parser/deserializer.ts Outdated Show resolved Hide resolved
test/types/deserialize.test-d.ts Outdated Show resolved Hide resolved
nbbeeken
nbbeeken previously approved these changes Nov 19, 2021
Copy link
Contributor

@nbbeeken nbbeeken left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!!!

src/parser/deserializer.ts Outdated Show resolved Hide resolved
@dariakp dariakp merged commit 07019a0 into master Nov 19, 2021
@dariakp dariakp deleted the NODE-3740 branch November 19, 2021 23:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Team Review Needs review from team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants