Skip to content

Commit

Permalink
Raise default --watchfs event limit
Browse files Browse the repository at this point in the history
By default, the JDK only reports up to 500 events per `WatchKey` (and thus, per directory). This can be too low in large repos.

Work towards bazelbuild#13226

RELNOTES: On Linux, the default limit on the number of `--watchfs` file events per directory has been raised to 10,000 (from 500). If needed, it can be increased further via `--host_jvm_args=-Djdk.nio.file.WatchService.maxEventsPerPoll=<limit>`.

Closes bazelbuild#23682.

PiperOrigin-RevId: 713570760
Change-Id: Ib3a61ad5d7cfaad4b7bbc25110f854e978e7fa75
  • Loading branch information
fmeum authored and copybara-github committed Jan 9, 2025
1 parent 457d248 commit 2280dba
Show file tree
Hide file tree
Showing 2 changed files with 37 additions and 0 deletions.
6 changes: 6 additions & 0 deletions src/main/cpp/blaze.cc
Original file line number Diff line number Diff line change
Expand Up @@ -426,6 +426,12 @@ static vector<string> GetServerExeArgs(const blaze_util::Path &jvm_path,
result.push_back("-Duser.language=");
result.push_back("-Duser.variant=");

// Allow more files to be watched per directory than the default limit of 500.
// The limit of 10,000 is arbitrary, but should be sufficient for most cases
// and can always be increased by the user if necessary.
// https://github.com/openjdk/jdk/blob/2faf8b8d582183275b1fdc92313a1c63c1753e80/src/java.base/share/classes/sun/nio/fs/AbstractWatchKey.java#L40
result.push_back("-Djdk.nio.file.WatchService.maxEventsPerPoll=10000");

if (startup_options.host_jvm_debug) {
BAZEL_LOG(USER)
<< "Running host JVM under debugger (listening on TCP port 5005).";
Expand Down
31 changes: 31 additions & 0 deletions src/test/shell/integration/watchfs_test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -143,4 +143,35 @@ EOF
assert_contains "bar" "${PRODUCT_NAME}-bin/pkg/foo.out"
}

function test_large_number_of_files() {
if [[ "${PLATFORM-}" == "darwin" ]]; then
# Tests Linux-specific JVM flags.
return 0
fi

local -r pkg=${FUNCNAME[0]}
mkdir $pkg || fail "mkdir $pkg"
cat > "$pkg/BUILD" << 'EOF'
genrule(
name = "gen",
outs = ["out"],
srcs = [":srcs"],
cmd = "touch $@",
)
filegroup(
name = "srcs",
srcs = glob(["*.txt"]),
)
EOF
touch "$pkg/1.txt"
bazel build --watchfs "//$pkg:gen" &> "$TEST_log" || fail "Expected success."

# By default, the JVM will only report up to 500 changed files per directory before it overflows.
# https://github.com/openjdk/jdk/blob/2faf8b8d582183275b1fdc92313a1c63c1753e80/src/java.base/share/classes/sun/nio/fs/AbstractWatchKey.java#L40
touch "$pkg/{1..600}.txt"
bazel build --watchfs "//$pkg:gen" &> "$TEST_log" || fail "Expected success."
expect_not_log "WARNING:"
}

run_suite "Integration tests for --watchfs."

0 comments on commit 2280dba

Please sign in to comment.