Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Downloaders for historical transfers #1473

Merged
merged 36 commits into from
Jun 13, 2019
Merged

Downloaders for historical transfers #1473

merged 36 commits into from
Jun 13, 2019

Commits on May 31, 2019

  1. Implement downloaders for historical transfers

    @dshulyak
    dshulyak committed May 31, 2019
    Configuration menu
    Copy the full SHA
    a0ef54e View commit details
    Browse the repository at this point in the history

  2. Store only hash and number of the block

    @dshulyak
    dshulyak committed May 31, 2019
    Configuration menu
    Copy the full SHA
    e214a73 View commit details
    Browse the repository at this point in the history

  3. Implement iterative downloader

    @dshulyak
    dshulyak committed May 31, 2019
    Configuration menu
    Copy the full SHA
    ff3355f View commit details
    Browse the repository at this point in the history

  4. Add tests that ensures that historical transfers will be included

    @dshulyak
    dshulyak committed May 31, 2019
    Configuration menu
    Copy the full SHA
    557be6e View commit details
    Browse the repository at this point in the history

  5. Send history event and update REDME

    @dshulyak
    dshulyak committed May 31, 2019
    Configuration menu
    Copy the full SHA
    654f1bf View commit details
    Browse the repository at this point in the history

  6. Add more logs and adjust timeout after debuggin on main chain

    @dshulyak
    dshulyak committed May 31, 2019
    Configuration menu
    Copy the full SHA
    84fea8b View commit details
    Browse the repository at this point in the history

Commits on Jun 1, 2019

  1. Use TransactionByHash instead of TransactionInBlock

    @dshulyak
    dshulyak committed Jun 1, 2019
    Configuration menu
    Copy the full SHA
    f671872 View commit details
    Browse the repository at this point in the history

  2. Adjust timeouts for erc20 downloader

    @dshulyak
    dshulyak committed Jun 1, 2019
    Configuration menu
    Copy the full SHA
    5e384d2 View commit details
    Browse the repository at this point in the history

  3. Transaction received in block can be nil

    @dshulyak
    dshulyak committed Jun 1, 2019
    Configuration menu
    Copy the full SHA
    ec95d81 View commit details
    Browse the repository at this point in the history

  4. Fix expectation in e2e test

    @dshulyak
    dshulyak committed Jun 1, 2019
    Configuration menu
    Copy the full SHA
    6a67f43 View commit details
    Browse the repository at this point in the history

  5. Check transaction.To after signer

    @dshulyak
    dshulyak committed Jun 1, 2019
    Configuration menu
    Copy the full SHA
    a383e85 View commit details
    Browse the repository at this point in the history

  6. Raise waiting interval if there was an error in any loop

    @dshulyak
    dshulyak committed Jun 1, 2019
    Configuration menu
    Copy the full SHA
    1fb7fc2 View commit details
    Browse the repository at this point in the history

Commits on Jun 2, 2019

  1. Accept hex encoding in API and handle not specified start value

    @dshulyak
    dshulyak committed Jun 2, 2019
    Configuration menu
    Copy the full SHA
    8f5e869 View commit details
    Browse the repository at this point in the history

Commits on Jun 3, 2019

  1. Apply several suggestions after review 

    - More debug info when node is stopped
- low keys in events
- send history event only if new transfers were found
    @dshulyak
    dshulyak committed Jun 3, 2019
    Configuration menu
    Copy the full SHA
    45fbd2e View commit details
    Browse the repository at this point in the history

Commits on Jun 13, 2019

  1. Implement faster downloader for historical eth transfers

    @dshulyak
    dshulyak committed Jun 13, 2019
    Configuration menu
    Copy the full SHA
    bdf72b0 View commit details
    Browse the repository at this point in the history

  2. Test binary iterative downloader

    @dshulyak
    dshulyak committed Jun 13, 2019
    Configuration menu
    Copy the full SHA
    7162816 View commit details
    Browse the repository at this point in the history

  3. Remove unused ethbatchSize

    @dshulyak
    dshulyak committed Jun 13, 2019
    Configuration menu
    Copy the full SHA
    f803c40 View commit details
    Browse the repository at this point in the history

  4. Use parent ctx in iterators and hash of tx as a unique identifier

    @dshulyak
    dshulyak committed Jun 13, 2019
    Configuration menu
    Copy the full SHA
    fa3974d View commit details
    Browse the repository at this point in the history

  5. Replace binary iterative downloader with concurrent downloader for eth

    @dshulyak
    dshulyak committed Jun 13, 2019
    Configuration menu
    Copy the full SHA
    7d66a23 View commit details
    Browse the repository at this point in the history

  6. Refactor logic from reactor

    @dshulyak
    dshulyak committed Jun 13, 2019
    Configuration menu
    Copy the full SHA
    f970755 View commit details
    Browse the repository at this point in the history

  7. Rework downloaders so that can accept multiple accounts

    @dshulyak
    dshulyak committed Jun 13, 2019
    Configuration menu
    Copy the full SHA
    38049c0 View commit details
    Browse the repository at this point in the history

  8. Support for downloading transfers for multiple accounts

    @dshulyak
    dshulyak committed Jun 13, 2019
    Configuration menu
    Copy the full SHA
    4fc000e View commit details
    Browse the repository at this point in the history

  9. Update readme with minor fixes

    @dshulyak
    dshulyak committed Jun 13, 2019
    Configuration menu
    Copy the full SHA
    7a7ec16 View commit details
    Browse the repository at this point in the history

  10. Refactor issues reported by linter

    @dshulyak
    dshulyak committed Jun 13, 2019
    Configuration menu
    Copy the full SHA
    f77ff62 View commit details
    Browse the repository at this point in the history

  11. Rewrite concurrent downloader and make erc20 downloader concurrent

    @dshulyak
    dshulyak committed Jun 13, 2019
    Configuration menu
    Copy the full SHA
    3a2a6f5 View commit details
    Browse the repository at this point in the history

  12. Generate unique ID for every transfer

    @dshulyak
    dshulyak committed Jun 13, 2019
    Configuration menu
    Copy the full SHA
    cf0660b View commit details
    Browse the repository at this point in the history

  13. Write better tests for reorg handling and improve handling description

    @dshulyak
    dshulyak committed Jun 13, 2019
    Configuration menu
    Copy the full SHA
    742750a View commit details
    Browse the repository at this point in the history

  14. Renaming leftover

    @dshulyak
    dshulyak committed Jun 13, 2019
    Configuration menu
    Copy the full SHA
    6750b7c View commit details
    Browse the repository at this point in the history

  15. Add logs and simplify lastKnownHeader

    @dshulyak
    dshulyak committed Jun 13, 2019
    Configuration menu
    Copy the full SHA
    9c5b3c8 View commit details
    Browse the repository at this point in the history

  16. Update accounts with all blocks in the set

    @dshulyak
    dshulyak committed Jun 13, 2019
    Configuration menu
    Copy the full SHA
    f612b46 View commit details
    Browse the repository at this point in the history

  17. Encrypt database with same password taht is used for pfs db

    @dshulyak
    dshulyak committed Jun 13, 2019
    Configuration menu
    Copy the full SHA
    e8adb6b View commit details
    Browse the repository at this point in the history

  18. Rename ConcurrentRunner to AtomicGroup

    @dshulyak
    dshulyak committed Jun 13, 2019
    Configuration menu
    Copy the full SHA
    3e59c07 View commit details
    Browse the repository at this point in the history

  19. Add fast catchup after app was offline for longer than 15 blocks 

    When downloader is started it will go through following procedure:
- verify that the last synced block is still in canonical chain
- run fast indexer for every account from last synced block to head of the chain
- start watching block by block
    @dshulyak
    dshulyak committed Jun 13, 2019
    Configuration menu
    Copy the full SHA
    27021ac View commit details
    Browse the repository at this point in the history

  20. Remove ununes isSequence

    @dshulyak
    dshulyak committed Jun 13, 2019
    Configuration menu
    Copy the full SHA
    860e69c View commit details
    Browse the repository at this point in the history

  21. Increase timeout in test: bootstrap procedure may take longer now

    @dshulyak
    dshulyak committed Jun 13, 2019
    Configuration menu
    Copy the full SHA
    faf9525 View commit details
    Browse the repository at this point in the history

  22. Remove unused code in database and adjust documentation

    @dshulyak
    dshulyak committed Jun 13, 2019
    Configuration menu
    Copy the full SHA
    4f02424 View commit details
    Browse the repository at this point in the history