Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Uncaught NumberFormatException caused by --experimental_execution_log_file path #14949

Closed
JaredNeil opened this issue Mar 3, 2022 · 2 comments
Assignees
Labels
P2 We'll consider working on this in future. (Assignee optional) team-Local-Exec Issues and PRs for the Execution (Local) team type: bug

Comments

@JaredNeil
Copy link
Contributor

Description of the problem:

Bazel sometimes crashes when the path passed to --experimental_execution_log_file contains a number greater than Long.MAX_VALUE (9223372036854775807). This can also happen if you set --execution_log_json_file or --execution_log_binary_file because a temp file is created for the execution log file and it will have a large number in it.
Stack Trace:

java.lang.NumberFormatException: For input string: "99999999999999999999999999999999"
        at java.base/java.lang.NumberFormatException.forInputString(Unknown Source)
        at java.base/java.lang.Long.parseLong(Unknown Source)
        at java.base/java.lang.Long.parseLong(Unknown Source)
        at com.google.devtools.build.lib.profiler.Profiler$JsonTraceFileWriter.getSortIndex(Profiler.java:983)
        at com.google.devtools.build.lib.profiler.Profiler$JsonTraceFileWriter.enqueue(Profiler.java:851)
        at com.google.devtools.build.lib.profiler.Profiler.logTask(Profiler.java:557)
        at com.google.devtools.build.lib.profiler.Profiler.logSimpleTask(Profiler.java:580)
        at com.google.devtools.build.lib.unix.UnixFileSystem$ProfiledNativeFileOutputStream.write(UnixFileSystem.java:555)
        at java.base/java.io.BufferedOutputStream.write(Unknown Source)
        at java.base/java.io.FilterOutputStream.write(Unknown Source)
        at com.google.devtools.build.lib.util.io.AsynchronousFileOutputStream.lambda$new$0(AsynchronousFileOutputStream.java:67)
        at java.base/java.lang.Thread.run(Unknown Source)

This happens because

  1. The Profiler tries to parse a number from every thread name
  2. The SpawnLogModule creates an AsynchronousFileOutputStream, which spawns a thread with the name of the file in the thread's name.

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

It's not deterministic because it depends on some async thread stuff, but I can get it to happend pretty regularly by building stuff with --experimental_execution_log_file=/tmp/exec99999999999999999999999999999999.log

Possible Fixes

  1. Check the length of the number before you try to parse it
  2. Catch the NumberFormatException exception and return MAX_SORT_INDEX;
  3. Verify the thread name is one we care about before parsing the number

If you have any preference as to which solution to implement, I'd be happy to submit a PR.

What operating system are you running Bazel on?

Ubuntu 20.04

What's the output of bazel info release?

release 5.0.0

@gregestren gregestren added team-Remote-Exec Issues and PRs for the Execution (Remote) team type: bug untriaged labels Mar 7, 2022
@coeuvre coeuvre added team-Local-Exec Issues and PRs for the Execution (Local) team and removed team-Remote-Exec Issues and PRs for the Execution (Remote) team labels Mar 21, 2022
@meisterT meisterT self-assigned this Apr 5, 2022
@meisterT meisterT added P2 We'll consider working on this in future. (Assignee optional) and removed untriaged labels Apr 5, 2022
@brentleyjones
Copy link
Contributor

@bazel-io flag

@bazel-io bazel-io added the potential release blocker Flagged by community members using "@bazel-io flag". Should be added to a release blocker milestone label Apr 6, 2022
@ckolli5
Copy link

ckolli5 commented Apr 13, 2022

@bazel-io fork 5.2.0

@bazel-io bazel-io removed the potential release blocker Flagged by community members using "@bazel-io flag". Should be added to a release blocker milestone label Apr 13, 2022
ckolli5 added a commit that referenced this issue May 9, 2022
Fixes #14949.

PiperOrigin-RevId: 439775241

Co-authored-by: twerth <twerth@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P2 We'll consider working on this in future. (Assignee optional) team-Local-Exec Issues and PRs for the Execution (Local) team type: bug
Projects
None yet
Development

No branches or pull requests

7 participants