Running in Batch mode barely passes any tests

[aniketp]

Run in 'batch mode' ? [y/N] y
Number of utilities to select for test generation: 5

           Utility | Progress
         ----------+-----------
        fingerd(8) | 5/5
echo '#! /usr/libexec/atf-sh' > fingerd_test.tmp
cat fingerd_test.sh >>fingerd_test.tmp
chmod +x fingerd_test.tmp
mv fingerd_test.tmp fingerd_test
install  -o root  -g wheel -m 555  fingerd_test  /usr/tests/libexec/fingerd/fingerd_test
install  -o root  -g wheel -m 444  Kyuafile  /usr/tests/libexec/fingerd/Kyuafile
fingerd_test:invalid_usage  ->  failed: atf-check failed; see the output of the test for details  [0.031s]
fingerd_test:no_arguments  ->  failed: atf-check failed; see the output of the test for details  [0.029s]

Results file id is usr_tests_libexec_fingerd.20180130-202309-467799
Results saved to /root/.kyua/store/results.usr_tests_libexec_fingerd.20180130-202309-467799.db

0/2 passed (2 failed)
          talkd(8) | 0/0
echo '#! /usr/libexec/atf-sh' > talkd_test.tmp
cat talkd_test.sh >>talkd_test.tmp
chmod +x talkd_test.tmp
mv talkd_test.tmp talkd_test
install  -o root  -g wheel -m 555  talkd_test  /usr/tests/libexec/talkd/talkd_test
install  -o root  -g wheel -m 444  Kyuafile  /usr/tests/libexec/talkd/Kyuafile
talkd_test:no_arguments  ->  failed: atf-check failed; see the output of the test for details  [0.056s]

Results file id is usr_tests_libexec_talkd.20180130-202309-795414
Results saved to /root/.kyua/store/results.usr_tests_libexec_talkd.20180130-202309-795414.db

0/1 passed (1 failed)
         rbootd(8) | 3/3
echo '#! /usr/libexec/atf-sh' > rbootd_test.tmp
cat rbootd_test.sh >>rbootd_test.tmp
chmod +x rbootd_test.tmp
mv rbootd_test.tmp rbootd_test
install  -o root  -g wheel -m 555  rbootd_test  /usr/tests/libexec/rbootd/rbootd_test
install  -o root  -g wheel -m 444  Kyuafile  /usr/tests/libexec/rbootd/Kyuafile
rbootd_test:invalid_usage  ->  failed: atf-check failed; see the output of the test for details  [0.014s]
rbootd_test:no_arguments  ->  failed: atf-check failed; see the output of the test for details  [0.014s]

Results file id is usr_tests_libexec_rbootd.20180130-202309-992132
Results saved to /root/.kyua/store/results.usr_tests_libexec_rbootd.20180130-202309-992132.db

0/2 passed (2 failed)
           ftpd(8) | 26/26
echo '#! /usr/libexec/atf-sh' > ftpd_test.tmp
cat ftpd_test.sh >>ftpd_test.tmp
chmod +x ftpd_test.tmp
mv ftpd_test.tmp ftpd_test
install  -o root  -g wheel -m 555  ftpd_test  /usr/tests/libexec/ftpd/ftpd_test
install  -o root  -g wheel -m 444  Kyuafile  /usr/tests/libexec/ftpd/Kyuafile
ftpd_test:invalid_usage  ->  failed: atf-check failed; see the output of the test for details  [0.013s]
ftpd_test:no_arguments  ->  failed: atf-check failed; see the output of the test for details  [0.014s]

Results file id is usr_tests_libexec_ftpd.20180130-202310-186159
Results saved to /root/.kyua/store/results.usr_tests_libexec_ftpd.20180130-202310-186159.db

0/2 passed (2 failed)
        rlogind(8) | 4/4
echo '#! /usr/libexec/atf-sh' > rlogind_test.tmp
cat rlogind_test.sh >>rlogind_test.tmp
chmod +x rlogind_test.tmp
mv rlogind_test.tmp rlogind_test
install  -o root  -g wheel -m 555  rlogind_test  /usr/tests/libexec/rlogind/rlogind_test
install  -o root  -g wheel -m 444  Kyuafile  /usr/tests/libexec/rlogind/Kyuafile
rlogind_test:invalid_usage  ->  failed: atf-check failed; see the output of the test for details  [0.013s]
rlogind_test:no_arguments  ->  failed: atf-check failed; see the output of the test for details  [0.013s]

Results file id is usr_tests_libexec_rlogind.20180130-202310-311523
Results saved to /root/.kyua/store/results.usr_tests_libexec_rlogind.20180130-202310-311523.db

0/2 passed (2 failed)

All the tests are failing on running the script in Batch mode. I tried this with multiple values "Number of Utilities" but to no benefit.

[shivansh]

Can you install and run an already existing test in the src tree (say echo_test.sh) and notify here what your results are.

[aniketp]

@shivansh, I installed the echo_test.sh in the corresponding location (/usr/src/bin/echo/tests) and made relevant changes to the parent and tests/ Makefiles. Tests were passing 👍

▶ kyua test                                                           
echo_test:append_c_output  ->  passed  [0.026s]
echo_test:n_output  ->  passed  [0.009s]

Results file id is usr_src_bin_echo_tests.20180301-203718-080360
Results saved to /root/.kyua/store/results.usr_src_bin_echo_tests.20180301-203718-080360.db

2/2 passed (0 failed)

[aniketp]

Though how was it related to the [Batch mode] testing or the observation above?