Failing commands in Windows Powershell & gitBash

[djthornton1212]

Note I'm not done with my test, it's just the end of the day for me right now and need to head out. I'll come back to this with anything more I find. Hopefully you'll do a better job at getting to the root of it than I have. Thanks!

Problem:

When attempting to run yarn run start after getOrgs & getRepos in PowerShell I received the following error:
Error: spawn mkdir ENOENT

When attempting to run yarn run start after getOrgs & getRepos gitBash I received the following error:
Error: spawn git ENOENT
Detailed logs below

System Info:

System OS: Windows
NodeJS: 17.5.0

Condig:

.env Config:
GITHUB_API_TOKEN=ghp_
GITHUB_ENTERPRISE=<private_company_name>
LANGUAGE_TO_CHECK=java
ENABLE_ON=secretscanning,dependabot,dependabotupdates,codescanning
CREATE_ISSUE=true

All other configs left at default

Logs:

When running in PowerShell in Windows 10

node:internal/process/promises:279
            triggerUncaughtException(err, true /* fromPromise */);
            ^

Error: spawn mkdir ENOENT
    at Process.ChildProcess._handle.onexit (node:internal/child_process:283:19)
    at onErrorNT (node:internal/child_process:476:16)
    at processTicksAndRejections (node:internal/process/task_queues:83:21) {
  errno: -4058,
  code: 'ENOENT',
  syscall: 'spawn mkdir',
  path: 'mkdir',
  spawnargs: [ '-p', 'tempGitLocations' ],
  cmd: 'mkdir -p tempGitLocations',
  stdout: '',
  stderr: ''
}

Node.js v17.5.0
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

---

When running in git-bash in Windows 10

node:internal/process/promises:279
            triggerUncaughtException(err, true /* fromPromise */);
            ^

Error: spawn git ENOENT
    at Process.ChildProcess._handle.onexit (node:internal/child_process:283:19)
    at onErrorNT (node:internal/child_process:476:16)
    at processTicksAndRejections (node:internal/process/task_queues:83:21) {
  errno: -4058,
  code: 'ENOENT',
  syscall: 'spawn git',
  path: 'git',
  spawnargs: [ 'checkout', '-b', 'ghas-Zigxs' ],
  cmd: 'git checkout -b ghas-Zigxs',
  stdout: '',
  stderr: ''
}

Node.js v17.5.0
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Expected behavior:

The GHAS tool should differentiate between CMD & PowerShell or specify one be used over the other.

I started to write out the commands here but quickly realized that's a lot to add lol. Suffice to say that though PowerShell can execute some commands from CMD, it seems to do a translation.
Example:
mkdir - Works in either case as there are no argument

rmdir - Doesn't translate over correctly.

• CMD: rmdir /s /q <path> - works fine

• PowerShell: rmdir /s /q <path> => Remove-Item: A positional parameter cannot be found that accepts argument '/q'.

Another I can see is rm. This isn't a command accepted in CMD but it is in PowerShell.

Troubleshooting:

In the end it appears that spawn is having issues detecting the commands to run. I know it's old, but it sees to still be an issue... maybe???
nodejs/node-v0.x-archive#2318

In short:

At the moment child_process.spawn() can only run exe files. This is a limitation of the CreateProcess API.

I tested this with the following:

..\ghas-enablement\serc\utils\commands.ts ->

const commands = [
    {
      command: "cmd",
      args: ["/s", "/c", "mkdir", "-p", `${tempDIR}`],
      cwd: `/Users/${winUser}/${windestDir}`,
    },
...

This actually worked (in PowerShell, but not CMD). I tried several permutations prior to this with no success.

Do you see the same error in your testing?

[NickLiffen]

Hey @djthornton1212 👍

Thanks for the feedback 🙇

So I don't have a Windows machine, and the only windows machine I would be able to test this on is a Windows Codespace. So this one may be tricky for me 😢

This should work on Windows though? But I don't know about Powershell 🤔

I will see if I can get a Windows machine and do some investigations.

The windows contributions were third party, I will see IF I can find anymore information 👍

[djthornton1212]

So, over the weekend I thought I'd give Windows Subsystem for Linux (WSL) a try and ran into the same situation.

npm notice 
npm notice New minor version of npm available! 8.5.5 -> 8.6.0
npm notice Changelog: https://github.com/npm/cli/releases/tag/v8.6.0
npm notice Run npm install -g npm@8.6.0 to update!
npm notice 
  ghas:inform Octokit Client Generated +0ms
  ghas:inform Currently looping over: 1/2. The org name is: <org> +42ms
  ghas:inform Currently looping over: 1/1. The repo name is: <org>/java-maven-junit-helloworld +43ms
  ghas:inform Enabled GHAS for java-maven-junit-helloworld. Status: 200 +992ms
  ghas:inform Is Dependabot Alerts enabled already for java-maven-junit-helloworld? : Enabled +458ms
  ghas:inform Enabled Dependabot Security Updates for java-maven-junit-helloworld. Status: 204 +450ms
  ghas:inform Enabled Secret Scanning for java-maven-junit-helloworld. Status: 200 +962ms
  ghas:inform Found default branch on the following repository: java-maven-junit-helloworld. The default branch is: main +377ms
  ghas:inform Found default branch SHA on the following repository: java-maven-junit-helloworld. The default branch is: 136169d210de3325020c47ed09bd0053239574af +315ms  ghas:inform Branch (ref) created on the following repository java-maven-junit-helloworld?. The branch reference is: refs/heads/ghas-AwHvI +485ms
  ghas:inform [
  ghas:inform   {
  ghas:inform     command: 'mkdir',
  ghas:inform     args: [ '-p', 'tempGitLocations' ],
  ghas:inform     cwd: '/Users/c/Desktop'
  ghas:inform   },
  ghas:inform   {
...
 ghas:error Error: spawn mkdir ENOENT
  ghas:error     at Process.ChildProcess._handle.onexit (node:internal/child_process:283:19)
  ghas:error     at onErrorNT (node:internal/child_process:476:16)
  ghas:error     at processTicksAndRejections (node:internal/process/task_queues:83:21) +0ms
node:internal/process/promises:279
            triggerUncaughtException(err, true /* fromPromise */);
            ^

Error: spawn mkdir ENOENT
    at Process.ChildProcess._handle.onexit (node:internal/child_process:283:19)
    at onErrorNT (node:internal/child_process:476:16)
    at processTicksAndRejections (node:internal/process/task_queues:83:21) {
  errno: -2,
  code: 'ENOENT',
  syscall: 'spawn mkdir',
  path: 'mkdir',
  spawnargs: [ '-p', 'tempGitLocations' ],
  cmd: 'mkdir -p tempGitLocations',
  stdout: '',
  stderr: ''
}

Node.js v17.9.0
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

My WSL is running Ubuntu 20.04.4 LTS (Focal Fossa).

[djthornton1212]

Ok, so I've re-written the windows commands to use cmd:

{
      command: "cmd",
      args: ["/s", "/c", "mkdir", `${tempDIR}`],
      cwd: `/Users/${winUser}/${windestDir}`,
    },
    {
      command: "cmd",
      args: ["/s", "/c", "git", "clone", `${baseURL}/${owner}/${repo}.git`],
      cwd: `/Users/${winUser}/${windestDir}/${tempDIR}`,
    },
    {
      command: "cmd",
      args: ["/s", "/c", "git", "checkout", "-b", `${branch}`],
      cwd: `/Users/${user}/${windestDir}/${tempDIR}/${repo}`,
    },

These work all the way up to the checkout step (last above). Here it continuously dies on me with:

ghas:error Error: spawn cmd ENOENT
ghas:error at Process.ChildProcess._handle.onexit (node:internal/child_process:283:19)
ghas:error at onErrorNT (node:internal/child_process:476:16)
ghas:error at processTicksAndRejections (node:internal/process/task_queues:83:21) +1s
node:internal/process/promises:279
triggerUncaughtException(err, true /* fromPromise */);
^

Error: spawn cmd ENOENT
at Process.ChildProcess._handle.onexit (node:internal/child_process:283:19)
at onErrorNT (node:internal/child_process:476:16)
at processTicksAndRejections (node:internal/process/task_queues:83:21) {
errno: -4058,
code: 'ENOENT',
syscall: 'spawn cmd',
path: 'cmd',
spawnargs: [ '/s', '/c', 'git', 'checkout', '-b', 'ghas-FpKHL' ],
cmd: 'cmd /s /c git checkout -b ghas-FpKHL',
stdout: '',
stderr: ''
}

When I navigate to the repository folder the app creates and manually copy/paste that command in (powershell) it works as expected. I'm not sure why it's failing here or where to go next.

I'm approaching my deadline to make this work. I might have to call it and start modifying these repositories by hand... yeah that's like 100+ repos.

---

Oh on a side note. You definitely should remove the -p from the mkdir commands. That's not a windows cmd command. What end up happening is 2 folders are created: tempGitLocations & -p.

It happens again when creating the .github/workflows folder. That's why you have to run:

{
      command: "cmd",
      args: ["/s", "/c", "rm", "-rf", '"./-p/"'],
      cwd: `/Users/${winUser}/${windestDir}/${tempDIR}/${repo}`,
    },
...
    {
      command: "cmd",
      args: ["/s", "/c", "rm", "-rf", '"./-p/"'],
      cwd: `/Users/${winUser}/${windestDir}`,
    },

later. That -p isn't a command but a folder being created 2x. Removing the -p negates the need to run those 2 commands.

[NickLiffen]

I'm approaching my deadline to make this work. I might have to call it and start modifying these repositories by hand... yeah that's like 100+ repos.

okay let's firstly fix this! the problem seems to be you are running on windows, can you use codespaces? There are docs for codespaces and it works from there. If you can't, can you access a Linux machine? I want to get you up and running before we try and fix the above windows commands (which obviously need solving)

@djthornton1212

[NickLiffen]

@djthornton1212 I would so test this on a Windows machine but I don't have any 😢

Have you looked at these articles:

1. Stackoverflow
2. GitHub Issue

I know they are all not directly related, but they are saying the same thing 🤔

[djthornton1212]

Hey Nick, thanks for the response.

My company is relatively new to GitHub and we haven't purchased codespaces at this point in time.

The first article I have seen, the second I had not. In either case the path to sysetm32 is set in PATH on my system.

I've tested this in CMD, PowerShell, Git Bash and my WSL Ubuntu installation. None can make it all the way through the commands.

CMD:

• Running the code as is it can't perform the first mkdir

Rrror: spawn mkdir ENOENT
at Process.ChildProcess._handle.onexit (node:internal/child_process:283:19)
at onErrorNT (node:internal/child_process:476:16)
at processTicksAndRejections (node:internal/process/task_queues:83:21) {
errno: -4058,
code: 'ENOENT',
syscall: 'spawn mkdir',
path: 'mkdir',
spawnargs: [ 'tempGitLocations' ],
cmd: 'mkdir tempGitLocations',
stdout: '',
stderr: ''
}

• Running the code after adding shell: true to execFile() it makes it to the git command checkout -b before failing

Error: spawn C:\windows\system32\cmd.exe ENOENT
at Process.ChildProcess._handle.onexit (node:internal/child_process:283:19)
at onErrorNT (node:internal/child_process:476:16)
at processTicksAndRejections (node:internal/process/task_queues:83:21) {
errno: -4058,
code: 'ENOENT',
syscall: 'spawn C:\windows\system32\cmd.exe',
path: 'C:\windows\system32\cmd.exe',
spawnargs: [ '/d', '/s', '/c', '"git checkout -b ghas-Gmlmh"' ],
cmd: 'git checkout -b ghas-Gmlmh',
stdout: '',
stderr: ''
}

PowerShell:

• Running the code as is it can't perform the first mkdir

Error: spawn mkdir ENOENT
at Process.ChildProcess._handle.onexit (node:internal/child_process:283:19)
at onErrorNT (node:internal/child_process:476:16)
at processTicksAndRejections (node:internal/process/task_queues:83:21) {
errno: -4058,
code: 'ENOENT',
syscall: 'spawn mkdir',
path: 'mkdir',
spawnargs: [ 'tempGitLocations' ],
cmd: 'mkdir tempGitLocations',
stdout: '',
stderr: ''
}

• Running the code after adding shell: true to execFile() it makes it to the git command checkout -b before failing

Error: spawn C:\windows\system32\cmd.exe ENOENT
at Process.ChildProcess._handle.onexit (node:internal/child_process:283:19)
at onErrorNT (node:internal/child_process:476:16)
at processTicksAndRejections (node:internal/process/task_queues:83:21) {
errno: -4058,
code: 'ENOENT',
syscall: 'spawn C:\windows\system32\cmd.exe',
path: 'C:\windows\system32\cmd.exe',
spawnargs: [ '/d', '/s', '/c', '"git checkout -b ghas-fbRTS"' ],
cmd: 'git checkout -b ghas-fbRTS',
stdout: '',
stderr: ''
}

Ubuntu WSL:

• Running the code as is it can't perform the first mkdir

Error: spawn mkdir ENOENT
at Process.ChildProcess._handle.onexit (node:internal/child_process:283:19)
at onErrorNT (node:internal/child_process:476:16)
at processTicksAndRejections (node:internal/process/task_queues:83:21) {
errno: -2,
code: 'ENOENT',
syscall: 'spawn mkdir',
path: 'mkdir',
spawnargs: [ '-p', 'tempGitLocations' ],
cmd: 'mkdir -p tempGitLocations',
stdout: '',
stderr: ''
}

• Running the code after adding shell: true to execFile() it makes it to the git command checkout -b before failing

Error: spawn /bin/sh ENOENT
at Process.ChildProcess._handle.onexit (node:internal/child_process:283:19)
at onErrorNT (node:internal/child_process:476:16)
at processTicksAndRejections (node:internal/process/task_queues:83:21) {
errno: -2,
code: 'ENOENT',
syscall: 'spawn /bin/sh',
path: '/bin/sh',
spawnargs: [ '-c', 'mkdir -p tempGitLocations' ],
cmd: 'mkdir -p tempGitLocations',
stdout: '',
stderr: ''
}

Question, I don't really see a commands section for linux. I know that the commands for macOS and ubuntu in this instance will likely be the same, but there are subtle difference like where the user's home directory is. Is this intentional? I believe my Ubuntu is failing because it's trying to make a directory in a location that doesn't exist and isn't really allowed to.

[djthornton1212]

Ok, so I went overboard and made a ...few changes. Got it working though. I'll push up my changes to my personal account so you can see what I did.

Short fix though, look at this line. That should be winUser.

Also, the rm command isn't a thing in CMD cli, it is however an alias in PowerShell.
Apparently, the following are all aliases to Remove-Item.

• ri
• rm
• rmdir
• del
• erase
• rd

If you really want to use rm you'll need to either use the CMD alias arguments: /Q /S or the PowerShell arguments: -recurse -force && it has to be initiated in PowerShell. In this regard I chose the route that exist in both rmdir /Q /S. This can be called in CMD as well as PowerShell.

[NickLiffen]

Short fix though, look at this line. That should be winUser.

I think we just merged in something which fixes this 👍

We also just added supported the wsl linux btw 👍 if you do a fresh pull you will get that 👍

Ok, so I went overboard and made a ...few changes. Got it working though. I'll push up my changes to my personal account so you can see what I did.

I would love this ❤️ that would be amazing 👍

[djthornton1212]

Hey Nick, sorry for the late response. I've posted what I've done here.

Please note that I'm relatively new to JavaScript/TypeScript (2nd project I've actually touched), so it's not pretty. There was a lot of googling "what does ... do" lol.

In the end the changes work for me. I was able to test with my single repo. I'm scheduling time to run against a larger pool size next.

[NickLiffen]

Please note that I'm relatively new to JavaScript/TypeScript (2nd project I've actually touched), so it's not pretty. There was a lot of googling "what does ... do" lol.

@djthornton1212 do not worry about this whatsoever 🙇 we have all been new so you shouldn't have to worry about not knowing!

In the end the changes work for me. I was able to test with my single repo. I'm scheduling time to run against a larger pool size next.

Great to hear 🙇

I've posted what I've done here.

Is there any chance you could make a pull request from your main branch into this projects main branch, so I am able to see what has changed 👍 that will allow me to review/give feedback/merge some stuff in 👍 I am sure what you have done is valuable for others and I would love to incorporate all the hard work into this repository 👍

[djthornton1212]

@NickLiffen I can do that. I'll merge my feature branching into main then open a PR to yours.

[djthornton1212]

Hey Nick, I wasn't sure if you saw that I created a PR for this.

[NickLiffen]

Hey @djthornton1212 I have been OOO 🙇 Today is my first day back, I am going to look at this now 👍

[djthornton1212]

@NickLiffen Great, I could use a vacay myself.

There are some conflicts from changes made after I started working on it. Let me know what you think.