Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dashboard should request log streams only when the user gestures to display them #3235

Merged
merged 13 commits into from
Apr 2, 2024

Conversation

eerhardt
Copy link
Member

@eerhardt eerhardt commented Mar 28, 2024

Change the AppHost to only subscribe to DCP's logs when there is a subscriber for the logs.

Fix #2789

Microsoft Reviewers: Open in CodeFlow

davidfowl and others added 4 commits March 26, 2024 18:54
Fix merge issue
…rs to not see what was already written.

Fix duplicate logs issue by clearing out the backlog when the last subscriber leaves.
@dotnet-issue-labeler dotnet-issue-labeler bot added the area-app-model Issues pertaining to the APIs in Aspire.Hosting, e.g. DistributedApplication label Mar 28, 2024
@JamesNK
Copy link
Member

JamesNK commented Mar 28, 2024

What happens to historical logs? For example, will requesting the console logs for an app show all longs since the app started? (if less than the limit)

…snap shotted after subscribing to the log.

Fix existing tests for new functionality and add an additional test.
@eerhardt
Copy link
Member Author

eerhardt commented Mar 28, 2024

What happens to historical logs? For example, will requesting the console logs for an app show all longs since the app started? (if less than the limit)

The "backlog" gets cleared after the last subscriber to a resource leaves. Later, when a "new" subscriber comes along, they are all replayed from DCP because we call ApplicationExecutor.StartLogStream(executable); again.

So even if no one was subscribed for a while, the whole logs come back once someone does.

@davidfowl
Copy link
Member

One small race is that starting logs before the container is ready, it won’t show up. Super tight race and I don’t know if it’ll be worth complicating the code but I thought I’d let you know

Only clear the backlog on containers and executables.
@eerhardt
Copy link
Member Author

One small race is that starting logs before the container is ready, it won’t show up.

Can you describe this a bit more? When what is starting the logs before the container is ready?

Move lock out of async iterator.
- Remove count and instead check if the delegate is null to indicate whether there are subscribers or not.
- Remove the separate IAsyncEnumerable classes and just use `async IAsyncEnumerable` methods.
Employ a 2nd loop that listens for both "has subscribers" and "logs available".
@eerhardt
Copy link
Member Author

One small race is that starting logs before the container is ready, it won’t show up. Super tight race and I don’t know if it’ll be worth complicating the code but I thought I’d let you know

This didn't turn out to be too bad. I added another loop that joined the "has subscribers" and "logs available" information, and only starts the logs once both are true.

eerhardt added 2 commits April 2, 2024 15:44
- Set SingleReader=true on the logInformationChannel.
- Add comment and assert that LogsAvailable can only turn true and can't go back to false.
@eerhardt eerhardt enabled auto-merge (squash) April 2, 2024 20:52
@eerhardt eerhardt disabled auto-merge April 2, 2024 20:53
@eerhardt eerhardt enabled auto-merge (squash) April 2, 2024 20:53
@eerhardt eerhardt merged commit a75513f into main Apr 2, 2024
8 checks passed
@eerhardt eerhardt deleted the on-demand-logs branch April 2, 2024 23:02
eerhardt added a commit to eerhardt/aspire that referenced this pull request Apr 2, 2024
…isplay them (dotnet#3235)

Change the AppHost to only subscribe to DCP's logs when there is a subscriber for the logs.

Fix dotnet#2789

* Show logs

* Remove the "StreamingLogger" because that causes subsequent subscribers to not see what was already written.

Fix duplicate logs issue by clearing out the backlog when the last subscriber leaves.

* Fix a concurrency issue with the backlog. Ensure the backlog is only snap shotted after subscribing to the log.

Fix existing tests for new functionality and add an additional test.

* PR feedback

Only clear the backlog on containers and executables.

* PR feedback.

Move lock out of async iterator.

* Clean up code.

- Remove count and instead check if the delegate is null to indicate whether there are subscribers or not.
- Remove the separate IAsyncEnumerable classes and just use `async IAsyncEnumerable` methods.

* Fix a race at startup when logs aren't available.

Employ a 2nd loop that listens for both "has subscribers" and "logs available".

* Simplify ResourceNotificationService.WatchAsync.

* Fix test build

* Address PR feedback

- Set SingleReader=true on the logInformationChannel.
- Add comment and assert that LogsAvailable can only turn true and can't go back to false.

---------

Co-authored-by: David Fowler <davidfowl@gmail.com>
radical pushed a commit to radical/aspire that referenced this pull request Apr 3, 2024
…isplay them (dotnet#3235)

Change the AppHost to only subscribe to DCP's logs when there is a subscriber for the logs.

Fix dotnet#2789

* Show logs

* Remove the "StreamingLogger" because that causes subsequent subscribers to not see what was already written.

Fix duplicate logs issue by clearing out the backlog when the last subscriber leaves.

* Fix a concurrency issue with the backlog. Ensure the backlog is only snap shotted after subscribing to the log.

Fix existing tests for new functionality and add an additional test.

* PR feedback

Only clear the backlog on containers and executables.

* PR feedback.

Move lock out of async iterator.

* Clean up code.

- Remove count and instead check if the delegate is null to indicate whether there are subscribers or not.
- Remove the separate IAsyncEnumerable classes and just use `async IAsyncEnumerable` methods.

* Fix a race at startup when logs aren't available.

Employ a 2nd loop that listens for both "has subscribers" and "logs available".

* Simplify ResourceNotificationService.WatchAsync.

* Fix test build

* Address PR feedback

- Set SingleReader=true on the logInformationChannel.
- Add comment and assert that LogsAvailable can only turn true and can't go back to false.

---------

Co-authored-by: David Fowler <davidfowl@gmail.com>
eerhardt added a commit that referenced this pull request Apr 4, 2024
…isplay them (#3235) (#3343)

Change the AppHost to only subscribe to DCP's logs when there is a subscriber for the logs.

Fix #2789

* Show logs

* Remove the "StreamingLogger" because that causes subsequent subscribers to not see what was already written.

Fix duplicate logs issue by clearing out the backlog when the last subscriber leaves.

* Fix a concurrency issue with the backlog. Ensure the backlog is only snap shotted after subscribing to the log.

Fix existing tests for new functionality and add an additional test.

* PR feedback

Only clear the backlog on containers and executables.

* PR feedback.

Move lock out of async iterator.

* Clean up code.

- Remove count and instead check if the delegate is null to indicate whether there are subscribers or not.
- Remove the separate IAsyncEnumerable classes and just use `async IAsyncEnumerable` methods.

* Fix a race at startup when logs aren't available.

Employ a 2nd loop that listens for both "has subscribers" and "logs available".

* Simplify ResourceNotificationService.WatchAsync.

* Fix test build

* Address PR feedback

- Set SingleReader=true on the logInformationChannel.
- Add comment and assert that LogsAvailable can only turn true and can't go back to false.

---------

Co-authored-by: David Fowler <davidfowl@gmail.com>
@danmoseley danmoseley mentioned this pull request Apr 12, 2024
@github-actions github-actions bot locked and limited conversation to collaborators May 3, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-app-model Issues pertaining to the APIs in Aspire.Hosting, e.g. DistributedApplication
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Dashboard should request log streams only when the user gestures to display them (for given resource)
3 participants