Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure the build check fails if there are new untracked files #77

Merged
merged 2 commits into from
Feb 15, 2022

Conversation

brrygrdn
Copy link
Contributor

@brrygrdn brrygrdn commented Feb 14, 2022

One gap in the current check run is that if a PR generates new files during a build which aren't checked as part of the PR, we ignore this.

This at least creates confusion as to why there is are untracked files locally when building on main, but in the worst case could cause us to fail to check in something that might be required in certain runtime environments.

Ignoring .node files generated by ssh2

We pull in ssh2 via dockerode:

dockerode
↳ docker-modem
  ↳ ssh2
    ↳ cpu-features

Both it and cpu-features compile optional native extensions using
OpenSSH and cpu-features (https://github.com/google/cpu_features) which
result in node-gyp generating two .node executables for these bindings

We have found that these bindings are very sensitive to the build env
which means that developer laptops, Actions and Codespaces result in
a diff in the .node files, something we seek to prevent in PRs in
order to detect cases where code changes are committed without actually
being applied to the dist/ folder.

Since we do not actually SSH into any containers in our implementation,
let's just ignore these files in our distributed code rather than
make our build more convoluted/less portable.

Why not just pin OpenSSH?

That was my first thought, but both Codespaces and Actions appear to be using the same base image of Ubuntu and OpenSSH/OpenSSL:

ssh -V
OpenSSH_8.2p1 Ubuntu-4ubuntu0.4, OpenSSL 1.1.1f 31 Mar 2020

I didn't want to get much further into the weeds of checking which developer lib versions where on both as the action currently works without these files being distributed, so they aren't involved in code paths we actually call.

@brrygrdn brrygrdn requested a review from a team as a code owner February 14, 2022 11:47
@brrygrdn
Copy link
Contributor Author

brrygrdn commented Feb 14, 2022

My expectation is that check dist/ verify build will fail as we currently have some untracked files on main's build. I'll add these in a follow-up commit so we have a clear red > green signal this is resolving an issue.

After investigation, I am not going to check in the .node files. I've updated the PR description accordingly.

@brrygrdn brrygrdn force-pushed the brrygrdn/update-check-diff-script branch 3 times, most recently from eba18cd to c586bb1 Compare February 15, 2022 13:21
Copy link
Member

@landongrindheim landongrindheim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Had a thought that I felt was very related to this change, but not within the scope of what you're doing. Feel free to ignore it 😄

@@ -3,6 +3,10 @@ output/output.json
repo
tmp/

# Ignore optional native extensions for SSH
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the context on this 😄

@@ -1,5 +1,7 @@
#!/bin/bash

# Make sure we notice any untracked files generated by the build
git add --intent-to-add .
Copy link
Member

@landongrindheim landongrindheim Feb 15, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This creeps outside the scope of the change you're introducing, but is related:

What do you think about replacing --quiet with --exit-code (on the line below this one)? My thinking is that we'd still trigger a build failure if the contents are different, but we'd be able to inspect the diff.

Copy link
Contributor Author

@brrygrdn brrygrdn Feb 15, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good call, I've modified the script to use --quiet, but in the event of a failure, output the full diff on line 8 (not shown in diff).

Not being able to see the diff was getting in my way a lot 😂

brrygrdn and others added 2 commits February 15, 2022 13:40
We pull in ssh2 via dockerode:

dockerode
↳ docker-modem
  ↳ ssh2
    ↳ cpu-features

Both it and `cpu-features` compile _optional_ native extensions using
OpenSSH and cpu-features (https://github.com/google/cpu_features) which
result in node-gyp generating two `.node` executables for these bindings

We have found that these bindings are very sensitive to the build env
which means that developer laptops, Actions and Codespaces result in
a diff in the `.node` files, something we seek to prevent in PRs in
order to detect cases where code changes are committed without actually
being applied to the `dist/` folder.

Since we do not actually SSH into any containers in our implementation,
let's just ignore these files in our distributed code rather than
make our build more convoluted/less portable.
@brrygrdn brrygrdn force-pushed the brrygrdn/update-check-diff-script branch from c586bb1 to 32688a2 Compare February 15, 2022 13:40
@brrygrdn brrygrdn merged commit c46e41a into main Feb 15, 2022
@brrygrdn brrygrdn deleted the brrygrdn/update-check-diff-script branch February 15, 2022 14:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants