Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(invariants): real-time runs counter #7302

Closed

Conversation

grandizzy
Copy link
Collaborator

@grandizzy grandizzy commented Mar 4, 2024

Motivation

proposed way to handle #585

multiple

Solution

  • add a global invariant reporter to collect test runs and completed runs
  • display in terminal on thread that polls data progress each 100ms
  • reporter is enabled if show_progress option is set to true (or run with env var FOUNDRY_INVARIANT_SHOW_PROGRESS=true forge test)

@grandizzy grandizzy changed the title Issue #585: invariant progress Issue #585: invariant run feedback Mar 4, 2024
@grandizzy grandizzy changed the title Issue #585: invariant run feedback feat invariant (Issue #585): invariant run feedback Mar 4, 2024
@grandizzy
Copy link
Collaborator Author

just noticed #3607 I think the approach of global invariant reporter could be a good foundation of adding such, wdyt?

Copy link
Member

@mattsse mattsse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is a very nice UX improvement, ty

pending @DaniPopes @klkvr

@DaniPopes
Copy link
Member

Have you tested this in the context of multiple other unit and fuzz tests? It looks like it will constantly get "overwritten" by the other test results that stream in, so I don't think this is right.

This is something I've thought about before, and I think I'll post a proposal as an issue soon and we'll discuss this internally soon.

Also I would prefer you follow conventional commits, at least for PR names 😸 (instead of feat abc (issue) you could write `feat(abc): do this thing)

@grandizzy
Copy link
Collaborator Author

grandizzy commented Mar 5, 2024

Have you tested this in the context of multiple other unit and fuzz tests? It looks like it will constantly get "overwritten" by the other test results that stream in, so I don't think this is right.

yep, I tested it, it is basically following the flow of results, see below

tests

This is something I've thought about before, and I think I'll post a proposal as an issue soon and we'll discuss this internally soon.

Also I would prefer you follow conventional commits, at least for PR names 😸 (instead of feat abc (issue) you could write `feat(abc): do this thing)

TIL, will do!

@grandizzy grandizzy changed the title feat invariant (Issue #585): invariant run feedback feat(invariant): real-time counter Mar 5, 2024
@grandizzy grandizzy changed the title feat(invariant): real-time counter feat(invariants): real-time runs counter Mar 5, 2024
@DaniPopes
Copy link
Member

Yeah you can see the spinner gets interleaved with the other results, I'd prefer we hold off on this for now.

That's a much better title, thanks!

@grandizzy
Copy link
Collaborator Author

grandizzy commented May 10, 2024

A different approach using indifcatif library
proposal

@grandizzy
Copy link
Collaborator Author

retired in favor of #7914

@grandizzy grandizzy closed this May 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants