Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problem: there are flaky integration tests #1244

Merged
merged 2 commits into from
Nov 15, 2023

Conversation

mmsqe
Copy link
Collaborator

@mmsqe mmsqe commented Nov 15, 2023

👮🏻👮🏻👮🏻 !!!! REFERENCE THE PROBLEM YOUR ARE SOLVING IN THE PR TITLE AND DESCRIBE YOUR SOLUTION HERE !!!! DO NOT FORGET !!!! 👮🏻👮🏻👮🏻

PR Checklist:

  • Have you read the CONTRIBUTING.md?
  • Does your PR follow the C4 patch requirements?
  • Have you rebased your work on top of the latest master?
  • Have you checked your code compiles? (make)
  • Have you included tests for any non-trivial functionality?
  • Have you checked your code passes the unit tests? (make test)
  • Have you checked your code formatting is correct? (go fmt)
  • Have you checked your basic code style is fine? (golangci-lint run)
  • If you added any dependencies, have you checked they do not contain any known vulnerabilities? (go list -json -m all | nancy sleuth)
  • If your changes affect the client infrastructure, have you run the integration test?
  • If your changes affect public APIs, does your PR follow the C4 evolution of public contracts?
  • If your code changes public APIs, have you incremented the crate version numbers and documented your changes in the CHANGELOG.md?
  • If you are contributing for the first time, please read the agreement in CONTRIBUTING.md now and add a comment to this pull request stating that your PR is in accordance with the Developer's Certificate of Origin.

Thank you for your code, it's appreciated! :)

Summary by CodeRabbit

  • Refactor
    • Improved the log retrieval process in integration tests by tracking the block number, ensuring more accurate and efficient testing.

Copy link
Contributor

coderabbitai bot commented Nov 15, 2023

Walkthrough

The update to the integration test script enhances the log tracking mechanism by introducing a start parameter. This parameter marks the block number from which to begin log retrieval, ensuring that logs are collected accurately and consistently after each message submission. This refinement helps maintain the integrity of the test flow and the reliability of the results.

Changes

File Path Change Summary
.../test_ica_precompile.py Added start parameter to wait_for_packet_log function to specify the starting block number for log retrieval. Updated start variable before each submit_msgs call.

Poem

In the code's autumn, a change takes flight, 🍂
A rabbit hops through the block's new light. 🐇💡
Logs align with a start so bright,
Ensuring the packets' journey is right. ✨


Tips

Chat with CodeRabbit Bot (@coderabbitai)

  • If you reply to a review comment from CodeRabbit, the bot will automatically respond.
  • To engage with CodeRabbit bot directly around the specific lines of code in the PR, mention @coderabbitai in your review comment
  • Note: Review comments are made on code diffs or files, not on the PR overview.
  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Commands (invoked as PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger a review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai help to get help.

Note: For conversation with the bot, please use the review comments on code diffs or files.

CodeRabbit Configration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • The JSON schema for the configuration file is available here.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/coderabbit-overrides.v2.json

@mmsqe mmsqe marked this pull request as ready for review November 15, 2023 05:52
@mmsqe mmsqe requested a review from a team as a code owner November 15, 2023 05:52
@mmsqe mmsqe requested review from calvinaco and thomas-nguy and removed request for a team November 15, 2023 05:52
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 0

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between cdfb5de and 8f16a82.
Files selected for processing (1)
  • integration_tests/test_ica_precompile.py (8 hunks)
Additional comments: 9
integration_tests/test_ica_precompile.py (9)
  • 180-191: The wait_for_packet_log function now accepts a start parameter to specify the starting block for log retrieval. This is a good improvement for performance and accuracy, as it narrows down the search space for logs, which can be particularly useful in a blockchain context where many blocks can be generated quickly. However, ensure that the start variable is correctly managed outside this function to avoid missing logs or searching from an incorrect block number.

  • 245-250: The submit_msgs_ro function is designed to test that read-only transactions fail as expected. This is a good practice for ensuring that the system behaves correctly in response to unauthorized actions. However, ensure that the status check is indeed intended to assert failure (status == 0) and is not a mistaken assertion.

  • 248-250: The start variable is updated to the current block number before calling submit_msgs. This is a crucial step to ensure that the log retrieval starts from the correct block after new transactions have been submitted. It's important to verify that this variable is updated at all necessary points in the test to maintain the accuracy of log retrieval.

  • 262-265: The wait_for_packet_log function is called after submitting messages to wait for a log with a specific sequence and status. This is a good use of the function to ensure that the expected events have occurred on-chain. However, ensure that the status variable reflects the actual status of the packet (e.g., Status.SUCCESS) and that the sequence number matches the expected sequence.

  • 284-287: Similar to the previous comment, ensure that the status and sequence are correctly asserted and that the wait_for_packet_log function is called with the correct parameters to wait for the expected log.

  • 307-310: Here, the test expects a Status.FAIL after submitting messages, which is likely testing the system's behavior under failure conditions. Ensure that the status is correctly set to Status.FAIL and that the wait_for_packet_log function is called with the correct parameters.

  • 331-334: Again, the test is expecting a Status.FAIL and waits for the corresponding log. It's important to ensure that the test setup is such that a failure is indeed expected at this point, and that the wait_for_packet_log function is called with the correct parameters.

  • 346-350: The register_acc function is called again, and it seems to be expected to return the same ica_address as before. This might be testing idempotency or that the registration process does not create duplicate accounts. Ensure that this behavior is expected and correctly implemented in the register_acc function.

  • 362-365: The test checks that the last_seq matches the expected_seq and that the status is Status.SUCCESS, then waits for the packet log. This is a good practice to ensure that the expected sequence of events has occurred. Ensure that the status and sequence are correctly asserted and that the wait_for_packet_log function is called with the correct parameters.

@mmsqe mmsqe enabled auto-merge November 15, 2023 06:16
@mmsqe mmsqe added this pull request to the merge queue Nov 15, 2023
Copy link

codecov bot commented Nov 15, 2023

Codecov Report

Merging #1244 (8f16a82) into main (cdfb5de) will decrease coverage by 20.23%.
The diff coverage is n/a.

Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff             @@
##             main    #1244       +/-   ##
===========================================
- Coverage   36.89%   16.67%   -20.23%     
===========================================
  Files         115       79       -36     
  Lines       10255     5786     -4469     
===========================================
- Hits         3784      965     -2819     
+ Misses       6097     4743     -1354     
+ Partials      374       78      -296     

see 54 files with indirect coverage changes

Merged via the queue into crypto-org-chain:main with commit 6834ef9 Nov 15, 2023
28 of 29 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants