Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add key type and duplicates verification with hashing #2245

Merged
merged 28 commits into from
May 10, 2021
Merged

Add key type and duplicates verification with hashing #2245

merged 28 commits into from
May 10, 2021

Commits on Apr 20, 2021

  1. Add hash_salt to ArrowWriter

    @NikhilBartwal
    NikhilBartwal committed Apr 20, 2021
    Configuration menu
    Copy the full SHA
    8b8ca4c View commit details
    Browse the repository at this point in the history

  2. Code quality changes

    @NikhilBartwal
    NikhilBartwal committed Apr 20, 2021
    Configuration menu
    Copy the full SHA
    318f740 View commit details
    Browse the repository at this point in the history

Commits on Apr 21, 2021

  1. Add key attribute to write()

    @NikhilBartwal
    NikhilBartwal committed Apr 21, 2021
    Configuration menu
    Copy the full SHA
    d7fbcdb View commit details
    Browse the repository at this point in the history

  2. Add keyhash.py

    @NikhilBartwal
    NikhilBartwal committed Apr 21, 2021
    Configuration menu
    Copy the full SHA
    2a5eac9 View commit details
    Browse the repository at this point in the history

  3. Add KeyHasher class and documentation

    @NikhilBartwal
    NikhilBartwal committed Apr 21, 2021
    Configuration menu
    Copy the full SHA
    95c9956 View commit details
    Browse the repository at this point in the history

  4. Fix filename typo

    @NikhilBartwal
    NikhilBartwal committed Apr 21, 2021
    Configuration menu
    Copy the full SHA
    8d8ff56 View commit details
    Browse the repository at this point in the history

Commits on Apr 24, 2021

  1. Add type-checking for keys

    @NikhilBartwal
    NikhilBartwal committed Apr 24, 2021
    Configuration menu
    Copy the full SHA
    976f1d2 View commit details
    Browse the repository at this point in the history

  2. Add hash method to KeyHasher

    @NikhilBartwal
    NikhilBartwal committed Apr 24, 2021
    Configuration menu
    Copy the full SHA
    b37ddfa View commit details
    Browse the repository at this point in the history

  3. Add KeyHasher to builder.py and ArrowWriter

    @NikhilBartwal
    NikhilBartwal committed Apr 24, 2021
    Configuration menu
    Copy the full SHA
    2aabb05 View commit details
    Browse the repository at this point in the history

  4. Fix KeyHasher bugs

    @NikhilBartwal
    NikhilBartwal committed Apr 24, 2021
    Configuration menu
    Copy the full SHA
    f6f16b4 View commit details
    Browse the repository at this point in the history

Commits on Apr 25, 2021

  1. Add duplicate key checking

    @NikhilBartwal
    NikhilBartwal committed Apr 25, 2021
    Configuration menu
    Copy the full SHA
    ec260a8 View commit details
    Browse the repository at this point in the history

  2. Minor bug fix

    @NikhilBartwal
    NikhilBartwal committed Apr 25, 2021
    Configuration menu
    Copy the full SHA
    d11c898 View commit details
    Browse the repository at this point in the history

Commits on Apr 28, 2021

  1. Making duplicate checking and keys optional

    @NikhilBartwal
    NikhilBartwal committed Apr 28, 2021
    Configuration menu
    Copy the full SHA
    c7766e0 View commit details
    Browse the repository at this point in the history

  2. Rearrange current_xamples structure

    @NikhilBartwal
    NikhilBartwal committed Apr 28, 2021
    Configuration menu
    Copy the full SHA
    4fbf53f View commit details
    Browse the repository at this point in the history

Commits on May 3, 2021

  1. Default key value to None 

    Co-authored-by: Quentin Lhoest <42851186+lhoestq@users.noreply.github.com>
    @NikhilBartwal @lhoestq
    NikhilBartwal and lhoestq authored May 3, 2021
    Configuration menu
    Copy the full SHA
    a8d1708 View commit details
    Browse the repository at this point in the history

  2. Merge master branch

    @NikhilBartwal
    NikhilBartwal committed May 3, 2021
    Configuration menu
    Copy the full SHA
    56400d2 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'hash_key_verification' of github.com:NikhilBartwal/data… 

    …sets-1 into hash_key_verification
    @NikhilBartwal
    NikhilBartwal committed May 3, 2021
    Configuration menu
    Copy the full SHA
    8fe744f View commit details
    Browse the repository at this point in the history

  4. Merge remote-tracking branch 'refs/remotes/upstream/master' into hash… 

    …_key_verification
    @NikhilBartwal
    NikhilBartwal committed May 3, 2021
    Configuration menu
    Copy the full SHA
    e084611 View commit details
    Browse the repository at this point in the history

  5. Update code-style

    @NikhilBartwal
    NikhilBartwal committed May 3, 2021
    Configuration menu
    Copy the full SHA
    f5261a4 View commit details
    Browse the repository at this point in the history

  6. Update black

    @NikhilBartwal
    NikhilBartwal committed May 3, 2021
    Configuration menu
    Copy the full SHA
    9dd361a View commit details
    Browse the repository at this point in the history

  7. Resolve isort errors

    @NikhilBartwal
    NikhilBartwal committed May 3, 2021
    Configuration menu
    Copy the full SHA
    98c0e85 View commit details
    Browse the repository at this point in the history

Commits on May 5, 2021

  1. Add test for invalid key datatype

    @NikhilBartwal
    NikhilBartwal committed May 5, 2021
    Configuration menu
    Copy the full SHA
    9b72afb View commit details
    Browse the repository at this point in the history

  2. Add test for checking duplicate keys

    @NikhilBartwal
    NikhilBartwal committed May 5, 2021
    Configuration menu
    Copy the full SHA
    d81f61c View commit details
    Browse the repository at this point in the history

  3. Code quality changes

    @NikhilBartwal
    NikhilBartwal committed May 5, 2021
    Configuration menu
    Copy the full SHA
    a522725 View commit details
    Browse the repository at this point in the history

Commits on May 7, 2021

  1. Merge branch 'master' of github.com:huggingface/datasets into hash_ke… 

    …y_verification
    @NikhilBartwal
    NikhilBartwal committed May 7, 2021
    Configuration menu
    Copy the full SHA
    4de5070 View commit details
    Browse the repository at this point in the history

  2. Apply suggestions from code review 

    Co-authored-by: Quentin Lhoest <42851186+lhoestq@users.noreply.github.com>
    @NikhilBartwal @lhoestq
    NikhilBartwal and lhoestq authored May 7, 2021
    Configuration menu
    Copy the full SHA
    17f150c View commit details
    Browse the repository at this point in the history

  3. Fix CI errors

    @NikhilBartwal
    NikhilBartwal committed May 7, 2021
    Configuration menu
    Copy the full SHA
    a2e1193 View commit details
    Browse the repository at this point in the history

  4. Add test for writing with keys

    @NikhilBartwal
    NikhilBartwal committed May 7, 2021
    Configuration menu
    Copy the full SHA
    8299b14 View commit details
    Browse the repository at this point in the history