Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't handle long strings #437

Closed
CaledoniaProject opened this issue Jul 28, 2023 · 5 comments · Fixed by #441
Closed

Can't handle long strings #437

CaledoniaProject opened this issue Jul 28, 2023 · 5 comments · Fixed by #441

Comments

@CaledoniaProject
Copy link

What should I do with bufio.scanner: token too long? Can you use a unbuffered reader?

@shueybubbles
Copy link
Collaborator

Can you please provide the steps to reproduce the issue? Is it with interactive mode or from an input file or both?

@CaledoniaProject
Copy link
Author

I just run sqlcmd -S XXX -C -i XXX.sql and it prints tons of bufio error. The ubuntu sqlcmd binary package does not suffer from this.

@shueybubbles
Copy link
Collaborator

For any size of xxx.sql? Or for a certain batch size inside the file?
What OS and sqlcmd version combination are you using?

@CaledoniaProject
Copy link
Author

CaledoniaProject commented Jul 29, 2023

I have completely no idea, because you're not printing the line number. I also think you don't have to know the size of the SQL file.

  1. You're using a fixed buffer size: https://github.com/microsoft/go-sqlcmd/blob/main/pkg/sqlcmd/sqlcmd.go#L42
  2. The solution is easily found on stackoverflow: https://stackoverflow.com/questions/21124327

If you want to produce the issue, you could create a large blob, e.g 100MB to test it. My input file is exported from a SQLServer. I believe that line is longer than your buffer size, unless there's a line number in the error output, I have no idea which line is the problem. But looks like at least 1000 lines is longer than that.

@shueybubbles
Copy link
Collaborator

thx for the pointer, we'll take a look

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants