Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Uninstall packages #21892

Merged
merged 33 commits into from
Sep 12, 2024
Merged

Uninstall packages #21892

merged 33 commits into from
Sep 12, 2024

Conversation

getvictor
Copy link
Member

@getvictor getvictor commented Sep 6, 2024

#20320

Demo video(s)

Checklist for submitter

  • Changes file added for user-visible changes in changes/, orbit/changes/ or ee/fleetd-chrome/changes.
    See Changes files for more information.
  • Input data is properly validated, SELECT * is avoided, SQL injection is prevented (using placeholders for values in statements)
  • Added/updated BE tests
  • Added/updated FE tests
  • If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes
  • If database migrations are included, checked table schema to confirm autoupdate
  • For database migrations:
    • Checked schema for all modified table for columns that will auto-update timestamps during migration.
    • Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects.
    • Ensured the correct collation is explicitly set for character columns (COLLATE utf8mb4_unicode_ci).
  • Manual BE QA for all new/changed functionality
  • Manual end-to-end QA for all new/changed functionality

Jacob Shandling and others added 17 commits September 3, 2024 15:45
…are modal (#21828)

## Addresses #21564 – see issue for task list
![Screenshot 2024-09-04 at 5 45
12 PM](https://github.com/user-attachments/assets/546401dd-b56e-4c39-baba-456dc844ee0f)
![Screenshot 2024-09-04 at 5 42
57 PM](https://github.com/user-attachments/assets/810ca450-0ddd-4258-96a5-bddb300ae19d)
![Screenshot 2024-09-04 at 5 45
02 PM](https://github.com/user-attachments/assets/32a19ce6-52c3-4772-ba53-00e50145bc85)
![Screenshot 2024-09-04 at 5 43
23 PM](https://github.com/user-attachments/assets/925843fb-6290-489b-a639-de1cbfba83fa)

- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
…stalling" states (#21858)

## #21565 
![Screenshot 2024-09-05 at 11 57
25 AM](https://github.com/user-attachments/assets/64e3ba67-482b-4e76-9e2f-ebce348bd104)
![Screenshot 2024-09-05 at 11 57
22 AM](https://github.com/user-attachments/assets/97c7ded5-a04d-4aac-bede-a7d65f8614b2)

- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
## Follow-up to #21564 
![Screenshot 2024-09-05 at 11 45
25 AM](https://github.com/user-attachments/assets/3efa3400-d600-4a9c-8488-40f497c6818c)
![Screenshot 2024-09-05 at 11 45
29 AM](https://github.com/user-attachments/assets/5919ba90-970b-4a51-a87d-5ed5226f1337)
![Screenshot 2024-09-05 at 11 45
32 AM](https://github.com/user-attachments/assets/f0ca80fe-fd80-4321-bd2a-8c33aa4c6f25)
![Screenshot 2024-09-05 at 11 45
35 AM](https://github.com/user-attachments/assets/84de54ad-8264-4f5a-a2cc-3aaf8fd3dbe1)

- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
# Conflicts:
#	server/datastore/mysql/schema.sql
Copy link

codecov bot commented Sep 6, 2024

Codecov Report

Attention: Patch coverage is 75.54981% with 189 lines in your changes missing coverage. Please review.

Project coverage is 65.33%. Comparing base (17edefe) to head (d9f7935).
Report is 56 commits behind head on main.

Files with missing lines Patch % Lines
...tallDetailsModal/SoftwareUninstallDetailsModal.tsx 8.82% 31 Missing ⚠️
ee/server/service/software_installers.go 77.39% 17 Missing and 9 partials ⚠️
...rations/tables/20240905200000_UninstallPackages.go 73.03% 13 Missing and 11 partials ⚠️
...ages/hosts/details/cards/Software/HostSoftware.tsx 0.00% 16 Missing ⚠️
...details/cards/Software/HostSoftwareTableConfig.tsx 0.00% 16 Missing ⚠️
...e/cards/ActivityFeed/ActivityItem/ActivityItem.tsx 7.14% 13 Missing ⚠️
server/datastore/mysql/scripts.go 84.90% 5 Missing and 3 partials ⚠️
server/datastore/mysql/software_installers.go 92.52% 4 Missing and 4 partials ⚠️
.../SoftwareInstallDetails/SoftwareInstallDetails.tsx 0.00% 7 Missing ⚠️
server/service/orbit.go 82.85% 3 Missing and 3 partials ⚠️
... and 14 more
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #21892      +/-   ##
==========================================
+ Coverage   64.96%   65.33%   +0.37%     
==========================================
  Files        1492     1493       +1     
  Lines      116151   116321     +170     
  Branches     3502     3540      +38     
==========================================
+ Hits        75454    75997     +543     
+ Misses      33692    33238     -454     
- Partials     7005     7086      +81     
Flag Coverage Δ
backend 66.65% <87.05%> (+0.45%) ⬆️
frontend 52.05% <15.32%> (-0.35%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@iansltx
Copy link
Member

iansltx commented Sep 7, 2024

Just so we don't forget, we still need the cron job to backfill uninstall scripts. Guessing that requires introspecting the entire installer, though maybe some shortcuts are available for some installer types?

I'm going to need to do a subset of this to determine "real" installer extension because we don't store that and I need it for Edit to validate that the installer type isn't changing as a result of the user uploading. Looks like I could just pull the first 64 bytes though and that would be enough information to determine the installer type.

@iansltx
Copy link
Member

iansltx commented Sep 7, 2024

Actually, potentially never mind on the optimizations...PE files look to have an offset value at 62 bytes, and then we have to look at what's at that offset. So I guess we pull everything.

Thoughts on adding an extension column in the DB as part of this work, and populating that as you're doing the uninstaller backfill? Grabbing every installer feels like a thing we want to do once...

@iansltx
Copy link
Member

iansltx commented Sep 7, 2024

Also seems like the uninstall script preprocess function would be easier to test/reuse if it specified a tighter interface and/or didn't change the script in-place? I'm probably overlooking why the way it is right now is better though.

@@ -89,6 +89,8 @@ type SoftwareInstaller struct {
InstallScript string `json:"install_script" db:"install_script"`
// InstallScriptContentID is the ID of the install script content.
InstallScriptContentID uint `json:"-" db:"install_script_content_id"`
// UninstallScriptContentID is the ID of the uninstall script content.
UninstallScriptContentID uint `json:"-" db:"uninstall_script_content_id"`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we also need UninstallScript here? We need uninstall script visible/editable in the places where we need e.g. InstallScript visible/editable.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

...yeah, 99.99% sure we do. Going to see if I can comit this to my branch so you can cherry-pick upstream if I did the work correctly.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3ec30ef should do the trick?

@getvictor
Copy link
Member Author

Just so we don't forget, we still need the cron job to backfill uninstall scripts.

I created a subtask for this work. It can be done as a separate PR. #21903

@getvictor
Copy link
Member Author

I'm going to need to do a subset of this to determine "real" installer extension because we don't store that and I need it for Edit to validate that the installer type isn't changing as a result of the user uploading.

I don't understand what we're validating here. We have the installer filename saved in software_installers table.

@getvictor
Copy link
Member Author

Also seems like the uninstall script preprocess function would be easier to test/reuse if it specified a tighter interface and/or didn't change the script in-place? I'm probably overlooking why the way it is right now is better though.

The PACKAGE_ID may not be correct. For example, a PKG installer may have 2 package IDs, but install only 1. So the user needs to be able to edit the actual package IDs that will be uninstalled.

iansltx and others added 2 commits September 8, 2024 12:20
Also hydrate uninstall script content ID when pulling installers when uninstall script contents aren't requested
# Conflicts:
#	server/datastore/mysql/schema.sql
#	server/fleet/software_installer.go
@iansltx iansltx mentioned this pull request Sep 10, 2024
3 tasks
Copy link
Member

@lucasmrod lucasmrod left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Submitting some questions/comments while I review tests.

return fleet.NewInvalidArgumentError("software_title_id", `No uninstall script exists for the provided "software_title_id".`).
WithStatus(http.StatusNotFound)
}
return err
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit ctxerr

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will fix

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I meant using ctxerr on line 707, but this is just a nit.

go.mod Outdated Show resolved Hide resolved
Comment on lines 15 to 21
# Start process and track exit code
$process = Start-Process @processOptions
$exitCode = $process.ExitCode

# Start process and track exit code
$process = Start-Process @processOptions
$exitCode = $process.ExitCode
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why twice?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe copy/paste or merge issue. Will fix.

# Get volume and location of package
volume=$(pkgutil --pkg-info "$pkg_id" | grep -i "volume" | awk '{for (i=2; i<NF; i++) printf $i " "; print $NF}')
location=$(pkgutil --pkg-info "$pkg_id" | grep -i "location" | awk '{for (i=2; i<NF; i++) printf $i " "; print $NF}')
# Check if this package id corresponds to a valid/installed package
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should it echo some warning if volume or location are empty?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, wouldn't hurt.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's an existing pkg/file/testdata/installers/ (empty with a .gitignore)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it is used by another test, so I couldn't put my file there. I didn't dig into the other test.

server/service/handler.go Show resolved Hide resolved
server/service/handler.go Show resolved Hide resolved
Comment on lines 267 to 269
si.uninstall_script_content_id,
si.uploaded_at,
si.uninstall_script_content_id,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why twice?

@@ -732,6 +800,12 @@ WHERE global_or_team_id = ?
}
installScriptID, _ := isRes.LastInsertId()

uisRes, err := insertScriptContents(ctx, tx, installer.UninstallScript)
if err != nil {
return ctxerr.Wrapf(ctx, err, "inserting install script contents for software installer with name %q", installer.Filename)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: inserting uninstall

Comment on lines 284 to 290
ExecAdhocSQL(t, ds, func(q sqlx.ExtContext) error {
_, err = q.ExecContext(ctx, `
UPDATE host_software_installs SET uninstall = 1 WHERE host_id = ? AND software_installer_id = ?`,
hostPendingUninstall.ID, si.InstallerID)
require.NoError(t, err)
return nil
})
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this done in ds.InsertSoftwareUninstallRequest above?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, will fix.

Copy link
Member

@lucasmrod lucasmrod left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Left a couple of questions.

ADD COLUMN uninstall_script_exit_code INT DEFAULT NULL,
ADD COLUMN uninstall TINYINT UNSIGNED NOT NULL DEFAULT 0,
ADD COLUMN status ENUM('pending_install', 'failed_install', 'installed', 'pending_uninstall', 'failed_uninstall')
GENERATED ALWAYS AS (
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for moving this logic as a generated column!

lucasmrod
lucasmrod previously approved these changes Sep 12, 2024
…activity feed, misc other items (#21933)

## #21566 - Host details updates for Uninstall packages

details > software page. Full tasks outlined in the issue, [Figma
here](https://www.figma.com/design/ToQaK2yUJwDyzagTdrbOfX/%2320320-Uninstall-packages?node-id=5364-13173&m=dev)

**Updated install status tooltips:**

![install-status-tooltips](https://github.com/user-attachments/assets/9869c7d6-f953-4adc-9692-52f5dad9d81a)

**Uninstall action:**

![uninstall-action](https://github.com/user-attachments/assets/189d5755-556c-48ca-8824-08db14ec95d4)

**Update install details:**
![Screenshot 2024-09-09 at 1 12
58 PM](https://github.com/user-attachments/assets/f52b349b-9f01-49d4-b952-6efd60f29979)


## #21931 - updated specs for install/uninstall states

## #21568 - activity feed items for Uninstall
![Screenshot 2024-09-09 at 5 00
07 PM](https://github.com/user-attachments/assets/eb61949a-9f8d-4b9e-a437-2d31a6808f07)
![Screenshot 2024-09-09 at 5 42
52 PM](https://github.com/user-attachments/assets/a8c2de0e-27e3-4d2b-bf69-702ea7b72e48)
![Screenshot 2024-09-09 at 5 43
03 PM](https://github.com/user-attachments/assets/b6127ed3-6fcf-439e-aa3d-91038a025d92)

## #21567 - Uninstall details modal
![Screenshot 2024-09-10 at 7 42
18 PM](https://github.com/user-attachments/assets/a42e4e4a-eadd-4e75-84c5-c5f6a6230950)
_remaining TODO_:
  - [x] manually QA 'failed' states
- [x] determine where to source timestamp from for uninstall activities

- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
Co-authored-by: Victor Lyuboslavsky <victor@fleetdm.com>
Copy link
Member

@lucasmrod lucasmrod left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-approving backend changes because latest commit/merge has no backend changes.

@getvictor getvictor marked this pull request as ready for review September 12, 2024 16:26
@getvictor getvictor requested review from rachaelshaw and a team as code owners September 12, 2024 16:26
@getvictor getvictor merged commit 8e5d056 into main Sep 12, 2024
29 checks passed
@getvictor getvictor deleted the 20320-feat-uninstall-packages branch September 12, 2024 16:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

6 participants