Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Source: files (s3 and local) #653

Merged
merged 24 commits into from
Dec 3, 2024
Merged

Source: files (s3 and local) #653

merged 24 commits into from
Dec 3, 2024

Commits on Dec 2, 2024

  1. initial commit, try to fix pickling error

    @tim-quix
    tim-quix committed Dec 2, 2024
    Configuration menu
    Copy the full SHA
    c66e2a5 View commit details
    Browse the repository at this point in the history

  2. fixed issue around pickling, need to simplify file gathering and part… 

    …ition counting
    @tim-quix
    tim-quix committed Dec 2, 2024
    Configuration menu
    Copy the full SHA
    6389fcd View commit details
    Browse the repository at this point in the history

  3. created more shared methods and unified classes more

    @tim-quix
    tim-quix committed Dec 2, 2024
    Configuration menu
    Copy the full SHA
    fa4ad96 View commit details
    Browse the repository at this point in the history

  4. a few more tweaks

    @tim-quix
    tim-quix committed Dec 2, 2024
    Configuration menu
    Copy the full SHA
    9045a00 View commit details
    Browse the repository at this point in the history

  5. move everything to file source

    @tim-quix
    tim-quix committed Dec 2, 2024
    Configuration menu
    Copy the full SHA
    6fee37d View commit details
    Browse the repository at this point in the history

  6. remove previous blob_store addition

    @tim-quix
    tim-quix committed Dec 2, 2024
    Configuration menu
    Copy the full SHA
    5dd5d98 View commit details
    Browse the repository at this point in the history

  7. clean up some things, add pagination handling to azure

    @tim-quix
    tim-quix committed Dec 2, 2024
    Configuration menu
    Copy the full SHA
    dae3fc6 View commit details
    Browse the repository at this point in the history

  8. add docs, rename some stuff

    @tim-quix
    tim-quix committed Dec 2, 2024
    Configuration menu
    Copy the full SHA
    211b3cd View commit details
    Browse the repository at this point in the history

  9. clean up schema doc section

    @tim-quix
    tim-quix committed Dec 2, 2024
    Configuration menu
    Copy the full SHA
    e23fc57 View commit details
    Browse the repository at this point in the history

  10. clean up compressor

    @tim-quix
    tim-quix committed Dec 2, 2024
    Configuration menu
    Copy the full SHA
    71792df View commit details
    Browse the repository at this point in the history

  11. tiny doc tweaks

    @tim-quix
    tim-quix committed Dec 2, 2024
    Configuration menu
    Copy the full SHA
    4c3e6c3 View commit details
    Browse the repository at this point in the history

  12. changes based on code review

    @tim-quix
    tim-quix committed Dec 2, 2024
    Configuration menu
    Copy the full SHA
    8cff599 View commit details
    Browse the repository at this point in the history

  13. Update docs/connectors/sources/aws-s3-file-source.md 

    Co-authored-by: Remy Gwaramadze <gwaramadze@users.noreply.github.com>
    @tim-quix @gwaramadze
    tim-quix and gwaramadze committed Dec 2, 2024
    Configuration menu
    Copy the full SHA
    8a699ac View commit details
    Browse the repository at this point in the history

  14. change bucket variable name

    @tim-quix
    tim-quix committed Dec 2, 2024
    Configuration menu
    Copy the full SHA
    c26a217 View commit details
    Browse the repository at this point in the history

  15. update local file docs

    @tim-quix
    tim-quix committed Dec 2, 2024
    Configuration menu
    Copy the full SHA
    bb12e07 View commit details
    Browse the repository at this point in the history

  16. add azure docs

    @tim-quix
    tim-quix committed Dec 2, 2024
    Configuration menu
    Copy the full SHA
    07af800 View commit details
    Browse the repository at this point in the history

  17. clean up azure stuff and other small tweaks

    @tim-quix
    tim-quix committed Dec 2, 2024
    Configuration menu
    Copy the full SHA
    72cba51 View commit details
    Browse the repository at this point in the history

  18. clean up after second round of code review

    @tim-quix
    tim-quix committed Dec 2, 2024
    Configuration menu
    Copy the full SHA
    f581e28 View commit details
    Browse the repository at this point in the history

  19. doc inclusions and cleanup

    @tim-quix
    tim-quix committed Dec 2, 2024
    Configuration menu
    Copy the full SHA
    4726928 View commit details
    Browse the repository at this point in the history

  20. some tweaks for consistency across source and sink

    @tim-quix
    tim-quix committed Dec 2, 2024
    Configuration menu
    Copy the full SHA
    5331152 View commit details
    Browse the repository at this point in the history

  21. adjust replay to only sleep with large enough deltas

    @tim-quix
    tim-quix committed Dec 2, 2024
    Configuration menu
    Copy the full SHA
    bd1bdff View commit details
    Browse the repository at this point in the history

Commits on Dec 3, 2024

  1. adjust replay speed to accept floats rather than a bool

    @tim-quix
    tim-quix committed Dec 3, 2024
    Configuration menu
    Copy the full SHA
    2c8483b View commit details
    Browse the repository at this point in the history

  2. remove azure stuff

    @tim-quix
    tim-quix committed Dec 3, 2024
    Configuration menu
    Copy the full SHA
    5a2e771 View commit details
    Browse the repository at this point in the history

  3. fix a doc ref

    @tim-quix
    tim-quix committed Dec 3, 2024
    Configuration menu
    Copy the full SHA
    e248f0c View commit details
    Browse the repository at this point in the history