Support os.proc on Scala Native

[lihaoyi]

Currently fails with

lihaoyi os-lib$ ./mill -i "os.native[2.12.17].test" test.os.SubprocessTests.lines
[229/229] os.native[2.12.17].test.test
[info] Starting process '/Users/lihaoyi/Github/os-lib/out/os/native/2.12.17/test/nativeLink.dest/out' on port '58844'.
----------------- Running Tests test.os.SubprocessTests.lines -----------------
X test.os.SubprocessTests.lines 70176ms
  java.lang.IllegalArgumentException
    scala.collection.immutable.VectorPointer.gotoNextBlockStartWritable(Unknown)
    scala.collection.immutable.VectorBuilder.gotoNextBlockStartWritable(Unknown)
    scala.collection.immutable.VectorBuilder.$plus$eq(Unknown)
    scala.collection.immutable.VectorBuilder.$plus$eq(Unknown)
    scala.collection.generic.Growable.$anonfun$$plus$plus$eq$1(Unknown)
    scala.collection.generic.Growable$$Lambda$1.apply(Unknown)
    scala.collection.Iterator.foreach(Unknown)
    scala.collection.AbstractIterator.foreach(Unknown)
    scala.collection.generic.Growable.$plus$plus$eq(Unknown)
    scala.collection.immutable.VectorBuilder.$plus$plus$eq(Unknown)
    scala.collection.immutable.VectorBuilder.$plus$plus$eq(Unknown)
    scala.collection.TraversableOnce.to(Unknown)
    scala.collection.AbstractIterator.to(Unknown)
    scala.collection.TraversableOnce.toIndexedSeq(Unknown)
    scala.collection.AbstractIterator.toIndexedSeq(Unknown)
    os.proc.call(Unknown)
    test.os.SubprocessTests$.$anonfun$tests$2(Unknown)
Tests: 1, Passed: 0, Failed: 1
1 targets failed
os.native[2.12.17].test.test 1 tests failed:
  test.os.SubprocessTests test.os.SubprocessTests.lines

[WojciechMazur]

Scala Native 0.5.1 is out, it should fix this issue

[lihaoyi]

Thanks @WojciechMazur !

I'm now hitting another issue trying to run ./mill -i "os.native[2.13.10].test" test.os.ListingWalkingTests.list.0 on this PR. Seems like Files.walkFileTree behaves differently between JVM and Native, and on Native it fails to enumerate all the files (e.g. out/scratch/ListingWalkingTests/tests/list/0/Multi Line.txt below) in the directory tree?

Native

visitFile out/scratch/ListingWalkingTests/tests/list/0/misc/file-symlink
visitFile out/scratch/ListingWalkingTests/tests/list/0/misc/broken-abs-symlink
visitFile out/scratch/ListingWalkingTests/tests/list/0/misc/broken-symlink
postVisitDirectory out/scratch/ListingWalkingTests/tests/list/0/misc
postVisitDirectory out/scratch/ListingWalkingTests/tests/list/0
X test.os.ListingWalkingTests.list.0 3ms
  java.nio.file.DirectoryNotEmptyException: out/scratch/ListingWalkingTests/tests/list/0
    java.nio.file.PosixException$.apply(Unknown)
    java.nio.file.Files$.$anonfun$unixDeletePath$1(Unknown)
    java.nio.file.Files$$$Lambda$3.apply(Unknown)
    scala.scalanative.unsafe.Zone$.acquire(Unknown)
    java.nio.file.Files$.unixDeletePath(Unknown)
    java.nio.file.Files$.delete(Unknown)
    java.nio.file.Files.delete(Unknown)
    test.os.TestUtil$$anon$1.postVisitDirectory(Unknown)
    test.os.TestUtil$$anon$1.postVisitDirectory(Unknown)
    java.nio.file.Files$._walkFileTree(Unknown)
    java.nio.file.Files$.walkFileTree(Unknown)
    java.nio.file.Files$.walkFileTree(Unknown)
    java.nio.file.Files.walkFileTree(Unknown)
    test.os.TestUtil$.prep(Unknown)
    test.os.ListingWalkingTests$.$anonfun$tests$3(Unknown)

Jvm

[188/188] os.jvm[2.13.10].test.test
--------------- Running Tests test.os.ListingWalkingTests.list.0 ---------------
visitFile out/scratch/ListingWalkingTests/tests/list/0/misc/file-symlink
visitFile out/scratch/ListingWalkingTests/tests/list/0/misc/broken-abs-symlink
visitFile out/scratch/ListingWalkingTests/tests/list/0/misc/broken-symlink
postVisitDirectory out/scratch/ListingWalkingTests/tests/list/0/misc
visitFile out/scratch/ListingWalkingTests/tests/list/0/Multi Line.txt
postVisitDirectory out/scratch/ListingWalkingTests/tests/list/0
+ test.os.ListingWalkingTests.list.0 1

[lihaoyi]

Minimal repro
scala-native/scala-native#3878

[lefou]

The issue should be fixed by:

• Fix #3878: javalib Files symlink deletion now matches JVM scala-native/scala-native#3887

[WojciechMazur]

It seems to pass using 0.5.2-SNAPSHOT, but there is a few other failures. I'll try to minimize and fix them later this week

[lihaoyi]

Thanks @WojciechMazur !

[WojciechMazur]

Update on the SN issues: The latest 0.5.2-SNAPSHOT is good enough to successfully run the tests thanks to:

• fix [javalib]: java.nio.file.Files.copy handling for symbolic links scala-native/scala-native#3913
• fix [javalib] - Improve handling of symbolic links in java.nio.file.Files scala-native/scala-native#3917
• fix [javalib]: Fix passing arguments to fallback /bin/sh -c command script in Unix process builders scala-native/scala-native#3918

There is however 1 ongoing issue related to Process Input/Output - the thread dedicated to processing stdout/stderr of process can fail with exception after process is finished due to bad file descriptor. I'm working on a fix to this problem. Anyway, if there is an urgent need we can publish a 0.5.2 this week with prepared fixes.

[lihaoyi]

Thanks @WojciechMazur! I think there's no urgent-urgent need at the moment. Looking forward to your fixes to scala-native so I can update the tests in the os-lib repo

[lolgab]

Scala Native 0.5.2 was just released which makes all tests pass 🎉

[lihaoyi]

Thanks @lolgab @WojciechMazur ! I guess this can be used without a new os-lib release, just by bumping the Scala-Native version right?

[lefou]

Thanks @lolgab @WojciechMazur ! I guess this can be used without a new os-lib release, just by bumping the Scala-Native version right?

I guess a release is needed, since you moved some source files from src-jvm to src.

[lihaoyi]

@lefou good point, I'll tag one

[lihaoyi]

I tagged OS-Lib 0.10.2, which should be out soon