On-prem: read blueprints from disk (COMPOSER-2413) #2603

kingsleyzissou · 2024-11-19T19:37:57Z

We can use the cockpit files api to read in blueprints from disk.

This PR adds commits to bring in the required dependencies, some convenience scripts and an initial commit using an RTK query function to read blueprints from file and pull the details into the store.

lucasgarfield · 2024-11-20T19:22:14Z

I can't wait to see this in action! I'm trying to get it running locally but can't get anything to appear in Cockpit. I noticed quite a few changes to the makefile... what should I do to get this to run?

kingsleyzissou · 2024-11-21T14:08:20Z

I can't wait to see this in action! I'm trying to get it running locally but can't get anything to appear in Cockpit. I noticed quite a few changes to the makefile... what should I do to get this to run?

Oh right, so we need to pull in some of the cockpit dependencies. Try make cockpit/devel, hopefully that does the trick. I am copying this all into a VM and then running that.

codecov · 2024-12-03T10:12:14Z

Codecov Report

Attention: Patch coverage is 8.21918% with 67 lines in your changes missing coverage. Please review.

Project coverage is 84.98%. Comparing base (9b69344) to head (5cca844).

Files with missing lines	Patch %	Lines
src/store/cockpitApi.ts	8.82%	31 Missing ⚠️
src/mocks/cockpit.ts	3.22%	30 Missing ⚠️
fec.config.js	0.00%	6 Missing ⚠️

@@            Coverage Diff             @@
##             main    #2603      +/-   ##
==========================================
- Coverage   85.23%   84.98%   -0.26%     
==========================================
  Files         175      176       +1     
  Lines       20253    20318      +65     
  Branches     1976     1977       +1     
==========================================
+ Hits        17263    17267       +4     
- Misses       2968     3029      +61     
  Partials       22       22

Files with missing lines	Coverage Δ
src/constants.ts	`100.00% <100.00%> (ø)`
fec.config.js	`0.00% <0.00%> (ø)`
src/mocks/cockpit.ts	`3.22% <3.22%> (ø)`
src/store/cockpitApi.ts	`35.48% <8.82%> (-22.85%)`	⬇️

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9b69344...5cca844. Read the comment docs.

kingsleyzissou · 2024-12-06T14:24:28Z

I had to change some typescript config which made this PR a lot bigger. I think if we want to go ahead with this, it would probably be better to split the typescript changes into a separate PR.

src/Components/Blueprints/BlueprintsSideBar.tsx

src/Components/ImagesTable/ImagesTableToolbar.tsx

lucasgarfield · 2024-12-10T19:22:53Z

It's working!

kingsleyzissou · 2024-12-11T17:34:36Z

/retest

The on-prem version of this repo depends on the `Cockpit` project. Some of the files from the `Cockpit` repo are needed to build and run this project. These files are saved into the `pkg/lib` directory, so we need ot add some plumbing for this into the project. We also need to add aliases for `cockpit` and `cockpit/fsinfo` modules. The downside is that we will need to create stub functions and types for the cockpit packages so typescript doesn't complain ¯\_(ツ)_/¯ Additionally, this commit adds some convenience scripts to the Makefile.

We need a toml package to parse the blueprint files, this appears to be the most maintained package.

Add an initial commit to scan a preset directory for user blueprint files. This makes use of the cockpit files api.

lucasgarfield · 2024-12-12T15:41:35Z

Makefile

@@ -1,17 +1,52 @@
 INSTALL_DIR := ~/.local/share/cockpit/image-builder-frontend

-cockpit/all: devel-uninstall devel-install build
+COCKPIT_REPO_URL = https://github.com/cockpit-project/cockpit.git
+COCKPIT_REPO_COMMIT = b0e82161b4afcb9f0a6fddd8ff94380e983b2238


We need a strategy for keeping this updated but we can figure that out later.

lucasgarfield · 2024-12-12T15:42:50Z

Makefile

+	sed -i "s/.*FIRST_BOOT_SERVICE_DATA.*/export const FIRST_BOOT_SERVICE_DATA = '$(FISTBOOT_SERVICE)';/" $@
+
+.PHONY: prep
+prep: src/constants.ts


Does this get used? Can we delete it?

lucasgarfield · 2024-12-12T15:43:22Z

Makefile

+.PHONY: prep
+prep: src/constants.ts
+
+.PHONY: install


lucasgarfield · 2024-12-12T15:43:27Z

Makefile

+install:
+	npm install
+
+.PHONY: start


lucasgarfield · 2024-12-12T15:51:21Z

fec.config.js

I'm a little worried here about co-mingling Cockpit "stuff" with Insights "stuff". Can we move things up one level to avoid this?

lucasgarfield · 2024-12-12T15:53:20Z

tsconfig.json

@@ -15,6 +15,11 @@
    ],
    "exactOptionalPropertyTypes": true,
    "strictNullChecks": true,
-    "allowSyntheticDefaultImports": true
+    "allowSyntheticDefaultImports": true,
+    "paths": {


Again, we have some co-mingling here. Maybe we can either eliminate this by moving everything up one level as previously suggested, or maybe we can have a base tsconfig.json and then extend it separately for Cockpit and Insights?

lucasgarfield · 2024-12-12T15:56:40Z

package.json

@@ -7,6 +7,7 @@
    "npm": ">=7.0.0"
  },
  "dependencies": {
+    "@ltd/j-toml": "1.38.0",


I'm not sure we actually need this. Sorry for not seeing this before and pointing it out but the Insights service already supports importing on-prem blueprints and I think there's already a "toml" package we can use.

lucasgarfield · 2024-12-12T15:58:11Z

src/mocks/cockpit.ts

+// TODO: maybe we should pull in the cockpit types here
+// and keep them in the project. Not ideal because it may
+// diverge, so we need to think about it.
+export interface UserInfo {


I need to set up the tslint to enforce this but generally I'm trying to use types and not interfaces.

kingsleyzissou requested a review from lucasgarfield November 19, 2024 19:37

lucasgarfield force-pushed the cockpit_fileapi branch from de7c995 to dcbf410 Compare November 20, 2024 13:34

kingsleyzissou force-pushed the cockpit_fileapi branch from dcbf410 to 091ff7d Compare November 20, 2024 14:27

kingsleyzissou force-pushed the cockpit_fileapi branch from 091ff7d to 12f3c12 Compare November 21, 2024 14:06

kingsleyzissou force-pushed the cockpit_fileapi branch 5 times, most recently from 2e17a88 to dadbaec Compare November 28, 2024 13:13

kingsleyzissou force-pushed the cockpit_fileapi branch 2 times, most recently from 6983959 to 85f4c58 Compare December 3, 2024 10:03

kingsleyzissou force-pushed the cockpit_fileapi branch 2 times, most recently from c73c587 to 8dd626a Compare December 4, 2024 15:43

lucasgarfield force-pushed the cockpit_fileapi branch from 8dd626a to b85d924 Compare December 5, 2024 19:16

kingsleyzissou force-pushed the cockpit_fileapi branch 7 times, most recently from 3640100 to 71de59a Compare December 6, 2024 14:13

lucasgarfield reviewed Dec 6, 2024

View reviewed changes

src/Components/Blueprints/BlueprintsSideBar.tsx Outdated Show resolved Hide resolved

lucasgarfield reviewed Dec 6, 2024

View reviewed changes

src/Components/ImagesTable/ImagesTableToolbar.tsx Outdated Show resolved Hide resolved

kingsleyzissou force-pushed the cockpit_fileapi branch from 71de59a to ad92853 Compare December 6, 2024 15:52

lucasgarfield changed the title ~~On-prem: read blueprints from disk~~ On-prem: read blueprints from disk (COMPOSER-2413) Dec 6, 2024

kingsleyzissou force-pushed the cockpit_fileapi branch from ad92853 to c14634f Compare December 9, 2024 15:12

schutzbot added the 🌟 best practice label Dec 9, 2024

kingsleyzissou force-pushed the cockpit_fileapi branch from c14634f to a5c5b94 Compare December 9, 2024 15:14

kingsleyzissou force-pushed the cockpit_fileapi branch from a5c5b94 to 0d662ae Compare December 11, 2024 16:46

kingsleyzissou added 3 commits December 12, 2024 09:38

package.json: add toml dependency

d06346c

We need a toml package to parse the blueprint files, this appears to be the most maintained package.

store/cockpitApi: scan for blueprint files

5cca844

Add an initial commit to scan a preset directory for user blueprint files. This makes use of the cockpit files api.

lucasgarfield force-pushed the cockpit_fileapi branch from 0d662ae to 5cca844 Compare December 12, 2024 15:38

lucasgarfield reviewed Dec 12, 2024

View reviewed changes

Makefile

.PHONY: prep

prep: src/constants.ts

.PHONY: install

Copy link

Collaborator

lucasgarfield Dec 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Delete.

lucasgarfield reviewed Dec 12, 2024

View reviewed changes

Makefile

install:

npm install

.PHONY: start

Copy link

Collaborator

lucasgarfield Dec 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Delete.

lucasgarfield reviewed Dec 12, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

On-prem: read blueprints from disk (COMPOSER-2413) #2603

On-prem: read blueprints from disk (COMPOSER-2413) #2603

kingsleyzissou commented Nov 19, 2024

lucasgarfield commented Nov 20, 2024

kingsleyzissou commented Nov 21, 2024

codecov bot commented Dec 3, 2024 •

edited

Loading

kingsleyzissou commented Dec 6, 2024

lucasgarfield commented Dec 10, 2024

kingsleyzissou commented Dec 11, 2024

lucasgarfield Dec 12, 2024

lucasgarfield Dec 12, 2024

lucasgarfield Dec 12, 2024

lucasgarfield Dec 12, 2024

lucasgarfield Dec 12, 2024

lucasgarfield Dec 12, 2024

lucasgarfield Dec 12, 2024

lucasgarfield Dec 12, 2024

On-prem: read blueprints from disk (COMPOSER-2413) #2603

Are you sure you want to change the base?

On-prem: read blueprints from disk (COMPOSER-2413) #2603

Conversation

kingsleyzissou commented Nov 19, 2024

lucasgarfield commented Nov 20, 2024

kingsleyzissou commented Nov 21, 2024

codecov bot commented Dec 3, 2024 • edited Loading

Codecov Report

kingsleyzissou commented Dec 6, 2024

lucasgarfield commented Dec 10, 2024

kingsleyzissou commented Dec 11, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Dec 3, 2024 •

edited

Loading