Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KAFKA-3968: fsync the parent directory of a segment file when the file is created #10405

Merged
merged 27 commits into from
Apr 3, 2021
Merged
Changes from 1 commit
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
459facb
[KAFKA-3968] fsync the parent directory of a segment file when the fi…
ccding Mar 25, 2021
b4c8284
move import
ccding Mar 26, 2021
b260169
address comments (except the topic partition one)
ccding Mar 29, 2021
ba086e9
remove import
ccding Mar 29, 2021
2a19e0e
reuse the function in utils.java
ccding Mar 30, 2021
40a1abe
simplify logic
ccding Mar 30, 2021
1ac80b6
default changeFileSuffixes flush to true
ccding Mar 30, 2021
09cac0b
flush when mkdirs
ccding Mar 30, 2021
5be95aa
revert accidential change
ccding Mar 30, 2021
c9448c8
atomicMoveWithFallback
ccding Mar 30, 2021
daeb698
no flush parent dir in test
ccding Mar 30, 2021
0d4800b
check null pointer
ccding Mar 31, 2021
95a6c3f
fix unit test error
ccding Mar 31, 2021
8c859f3
set flag after flush
ccding Apr 1, 2021
fdc1faa
disable flushing on renameTo
ccding Apr 1, 2021
6795ec9
address comments based on offline discussion with Jun
ccding Apr 1, 2021
55ae3bc
Merge branch 'trunk' into fsync
ccding Apr 1, 2021
e653af4
check hadCleanShutdown for open FileRecord
ccding Apr 1, 2021
85861ee
address comments
ccding Apr 1, 2021
1578678
fix default values
ccding Apr 1, 2021
fffc353
more default value
ccding Apr 1, 2021
f66c545
do flush in the LogSegment class
ccding Apr 2, 2021
56be9d8
Merge branch 'trunk' into fsync
ccding Apr 2, 2021
1ecf94b
remove parameter from FileRecord.open
ccding Apr 2, 2021
080a79a
default to false
ccding Apr 2, 2021
61eee4a
add param to javadoc
ccding Apr 2, 2021
7543938
during flush -> during the next flush
ccding Apr 2, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix unit test error
  • Loading branch information
ccding committed Mar 31, 2021
commit 95a6c3fe5aa4be1d366e36ac56b946fd9984022a
Original file line number Diff line number Diff line change
@@ -942,8 +942,6 @@ public static void flushParentDir(Path path) throws IOException {
dir = FileChannel.open(parent, StandardOpenOption.READ);
dir.force(true);
}
} catch (Exception e) {
throw new KafkaException("Attempt to flush the parent directory of " + path + " failed.");
} finally {
if (dir != null)
dir.close();