Mask getBasicFileAttributes errors in Files[F].walk

[fthomas]

This shows a problem with Files[F].walk where it does not walk the whole file tree if files are deleted during the process. The added tests fails with:

==> X fs2.io.file.FilesSuite.walk - can delete files in a nested tree  0.029s munit.ComparisonFailException: fs2/io/shared/src/test/scala/fs2/io/file/FilesSuite.scala:537
536:        .foldMonoid
537:        .assertEquals(25)
538:    }
values are not the same
=> Obtained
5
=> Diff (- obtained, + expected)
-5
+25

Only 5 files out of 25 where deleted. I expected that the whole process deletes 25 files because tempFilesHierarchy creates 5 child directories with 5 files in each child directory.

If Files[IO].deleteIfExists(path).as(1) is replaced with IO.pure(1) the test passes.

[armanbilge]

Thanks for the reproducer! So the issue is partly related to this .mask, which swallows the exception and stops iterating over the directory.

fs2/io/shared/src/main/scala/fs2/io/file/Files.scala

Lines 472 to 475 in 134041f

	if (attr.isDirectory)
	list(start).flatMap { path =>
	go(path, maxDepth - 1, attr.fileKey.toRight(start) :: ancestry)
	}.mask

If I take it away I get this.

==> X fs2.io.file.FilesSuite.walk - can delete files in a nested tree  0.482s java.nio.file.NoSuchFileException: /tmp/BaseFileSpec9152885937877967676/BaseFileSpec10790971575620310752/BaseFileSpecSub14584697728072684076.tmp

[armanbilge]

Ah yes, more masking seems to have solved it 😂

[diesalbla]

I wonder if some of this could be contained within the F effect itself, as there is no much use for streaming features:

Suggested change

	Stream.eval(getBasicFileAttributes(start, followLinks = true)).mask.flatMap {
	attr =>
	val fileKey = attr.fileKey
	val isCycle = Traverse[List].existsM(ancestry) {
	case Right(ancestorKey) => F.pure(fileKey.contains(ancestorKey))
	case Left(ancestorPath) => isSameFile(start, ancestorPath)
	}
	Stream.eval(isCycle).flatMap { isCycle =>
	if (!isCycle)
	list(start).mask.flatMap { path =>
	go(path, maxDepth - 1, attr.fileKey.toRight(start) :: ancestry)
	}
	else
	Stream.raiseError(new FileSystemLoopException(start.toString))
	}
	getBasicFileAttributes(start, followLinks = true).handleError(_ => ()).flatMap {
	attr =>
	val fileKey = attr.fileKey
	val isCycle = Traverse[List].existsM(ancestry) {
	case Right(ancestorKey) => F.pure(fileKey.contains(ancestorKey))
	case Left(ancestorPath) => isSameFile(start, ancestorPath)
	}
	isCycle.flatMap {
	case false =>
	list(start).mask.flatMap { path =>
	go(path, maxDepth - 1, attr.fileKey.toRight(start) :: ancestry)
	}
	case true => F.raiseError(new FileSystemLoopException(start.toString))
	}

[fthomas]

go is a Stream[F, Path] and you flatMap from F to it. I don't think this compiles.