Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When Stream() returns many lines, sometimes chunks of lines are not captured. #15

Closed
DiscoRiver opened this issue Oct 7, 2021 · 3 comments · Fixed by #16
Closed

When Stream() returns many lines, sometimes chunks of lines are not captured. #15

DiscoRiver opened this issue Oct 7, 2021 · 3 comments · Fixed by #16
Assignees
Labels
bug Something isn't working help wanted Extra attention is needed

Comments

@DiscoRiver
Copy link
Owner

Intermittently, running stream to capture a reasonable amount of text sometimes misses chunks of lines.

You can see an example of this here in "Go Test": https://github.com/DiscoRiver/massh/runs/3832011162?check_suite_focus=true

I was using the command cat /var/log/auth.log in this test, and you'll see the following on line 48;

do: pam_unix(sudo:session): session opened for user root by (uid=0)

This should be something like what is on line 44;

Oct  7 20:45:48 fv-az224-778 sudo: pam_unix(sudo:session): session opened for user root by (uid=0)

There doesn't immediately seem to be a pattern, but more investigation is necessary into the concurrency model.

@DiscoRiver DiscoRiver added the bug Something isn't working label Oct 7, 2021
@DiscoRiver DiscoRiver self-assigned this Oct 7, 2021
@DiscoRiver
Copy link
Owner Author

Confirmed that this does not happen with bulk return using massh.Config.Run()

@DiscoRiver DiscoRiver added the help wanted Extra attention is needed label Oct 8, 2021
@DiscoRiver
Copy link
Owner Author

Looks like I set the stream to work in chunks of 1024 bytes here, which unfortunately causes a few problems as we're no longer reading line-by-line.

Now I know the cause, I will investigate a more robust way of retaining data integrity on the output channels.

@DiscoRiver
Copy link
Owner Author

The function readToBytesChannel, which processes the Stdout and Stderr stream, has been updated to read lines for better granularity and lower complexity.

A wait group has also been added to ensure that all goroutines are returned elegantly. There was a problem with readToBytesChannel causing an issue further up the chain of execution, meaning each session was "completed" before all output had been processed.

Usage of this package should not be affected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working help wanted Extra attention is needed
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant