Suppress NoSuchFileException in LocalFileIterator

[ebyhr]

Description

Files.walk method may throw an exception during the iteration. https://bugs.openjdk.org/browse/JDK-8039910
Fixes #20520

Release notes

(x) This is not user-visible or is docs only, and no release notes are required.

[dain]

Please get @electrum to review this before merging. The exact behavior of exceptions from the file system implementations is required to be consistent, and I'd like david to make sure this is following the expected behavior.

[electrum]

The Javadoc for listFiles() says

If the location does not exist, an empty iterator is returned.

For hierarchical file systems, if the path is not a directory, an exception is raised.

The expectation is that it only throws in the above case, or if there is a network error or similar. We should never see NoSuchFileException in FileHiveMetastore for listFiles(). If we do, that's a bug in the file system implementation.

[electrum]

So I believe that the first commit is correct and the second commit is not needed.

[ebyhr]

@electrum Both commits are required to fix the concurrent issue. LocalFileIterator#next calls Files.size(path). It throws NoSuchFileException if the file is deleted concurrently.

[electrum]

Thanks for explaining. We should change LocalFileIterator to ignore those exceptions by not returning the file entry. Listing files is not atomic, so this is an expected race condition that we need to handle.

[findinpath]

We should change LocalFileIterator to ignore those exceptions by not returning the file entry.

@ebyhr did you consider making use of a simplified version of java.nio.file.Files#walkFileTree(java.nio.file.Path, java.util.Set<java.nio.file.FileVisitOption>, int, java.nio.file.FileVisitor<? super java.nio.file.Path>) (what Files.walkFileTree uses in the background) and create the FileEntry instances on the fly in the LocalFileIterator?