feat: proxy & browser capabilities #2

Llandy3d · 2024-06-03T14:40:11Z

Events

Introduce the logic for handling both the proxy and the browser via events.
Events are prefixed by the resource they work on.

Proxy Events:

proxy:start
proxy:started
proxy:stop
proxy:data: data received by the proxy, representing requests/responses

Browser Events:

browser:start
browser:started
browser:stop

The events are separated to maintain fine control and allow for future optimizations (for example, we can start the proxy before the user actually uses it, improving startup time)

Binaries

The binaries for the proxy are under the path resources, they will be retrieved later on depending on the os/architecture combination via process.resourcePath.

Certificates

The certificate for the mitm proxying is created by the proxy on the user machine when run for the first time. We retrieve this file and generate the SPKI to pass to the browser for accepting it. This allows us to avoid shipping a pre-created certificate that could be misused.

Example
index.html & renderer.ts showcase how to use those events for dealing with both the proxy & browser, it should be fairly straightforwrd.

json_output.py is the logic converting the proxy captured data into json for passing it to the node process, it will still need more work for handling different mime-types.

Types

For the types of the bridged context in window I've opted for the automatic approach, this should speedup development initially, but we can consider swapping back to manual typing 🤔

https://github.com/grafana/k6-cloud/issues/2386

Llandy3d · 2024-06-03T14:40:50Z

forge.config.ts


 const config: ForgeConfig = {
  packagerConfig: {
    asar: true,
+    extraResource: [
+      './resources/json_output.py',
+      './resources/' + getPlatform() + '/' + getArch(),


we will pick up only the specific os/architecture combination when packaging for a specific system

Llandy3d · 2024-06-03T14:41:30Z

src/electron.d.ts

@@ -0,0 +1,3 @@
+declare interface Window {
+    studio: import('../src/preload').Studio;
+}


this is the trick for not having to specify manually every type

Llandy3d · 2024-06-03T14:42:04Z

src/lib/types.ts

+  id: string,
+  request: Request,
+  response?: Response,
+}


this has more data but like this is enough to get us started

Llandy3d · 2024-06-03T22:01:54Z

resources/json_output.py

+                content_hash = None
+
+            # we base64 encode content and let the client deal with it depending on mimetype
+            content = base64.b64encode(flow.response.content).decode() if flow.response.content else None


using base64 encoding for "stringifying" binary data since it doesn't play well with the json format. This should allow us to handle all mime-types and eventually showcase them in a better way 😃

e-fisher

LGTM 🚀

going-confetti

Left a few comments, mostly questions/nitpicks, but with 2 highlighting potential bugs in types

going-confetti · 2024-06-04T08:48:37Z

src/browser.ts

+const createUserDataDir = async () => {
+    return await mkdtemp(path.join(os.tmpdir(), 'k6-studio-'));
+};


Nit: async and await are redundant here as long as mkdtemp returns a promise.

going-confetti · 2024-06-04T08:53:24Z

src/browser.ts

+  const userDataDir = await createUserDataDir();
+  console.log(userDataDir);
+  const certificateSPKI = await getCertificateSPKI();
+  const disableChromeOptimizations = "--disable-features=OptimizationGuideModelDownloading,OptimizationHintsFetching,OptimizationTargetPrediction,OptimizationHints";


Nit/future improvement: IMO having an array of disabled features, that can then be used to create this string will make it a bit more readable and easier to maintain:

// Could probably be an export from src/constants.ts or some other file with constants, but it's not super important const DISABLED_CHROME_FEATURES = [ "OptimizationGuideModelDownloading", "OptimizationHintsFetching", "OptimizationTargetPrediction", "OptimizationHints", ] // ... const disableChromeOptimizations = `--disable-features=${DISABLED_CHROME_FEATURES.join(',')}`

I will add a TODO, I also would like to have every flag passed to the chromium browsers to be documented so that we know the "why" they are there

going-confetti · 2024-06-04T08:57:32Z

src/lib/types.ts

+  status_code: number,
+  content: string,
+  path: string,
+  timestamp_start: Date,


Date is not serializable. It should probably be string

going-confetti · 2024-06-04T08:57:37Z

src/lib/types.ts

+  method: string,
+  path: string,
+  content: string,
+  timestamp_start: Date,


Date is not serializable. It should probably be string

Llandy3d · 2024-06-04T09:21:34Z

src/preload.ts

+
+const proxy = {
+  launchProxy: () => {
+    ipcRenderer.send('proxy:start');


I think this is an actually confusing bit about electron. As of today per my understanding you have both the .send()/.on() and .invoke()/.handle() methods combination.

For single direction communication is totally fine to use .send() and it's actually more explicit because you are saying that you are not expecting a response.

For bi-directional communication, although you can still do the old pattern with .send() it is recommended to use the .invoke() method when you expect a response back.

So .send() it's not a legacy approach on every scenario but just in the bi-directional communication part (where you find the warning).

going-confetti · 2024-06-04T09:08:02Z

src/lib/types.ts

+export interface Response {
+  headers: Array<Array<string>>;
+  reason: string,
+  status_code: number,


Question: At this point, we're used to snake_case on the frontend, but camelCase is way more common in the JS/TS world, so maybe we can consider it?

yes! This represents data from the proxy exposed in this way, we can try to change the casing either now or in Phase 2 🤔

Llandy3d · 2024-06-04T09:21:34Z

src/preload.ts

+
+const proxy = {
+  launchProxy: () => {
+    ipcRenderer.send('proxy:start');


I think this is an actually confusing bit about electron. As of today per my understanding you have both the .send()/.on() and .invoke()/.handle() methods combination.

For single direction communication is totally fine to use .send() and it's actually more explicit because you are saying that you are not expecting a response.

For bi-directional communication, although you can still do the old pattern with .send() it is recommended to use the .invoke() method when you expect a response back.

So .send() it's not a legacy approach on every scenario but just in the bi-directional communication part (where you find the warning).

going-confetti

LGTM 🚀

Llandy3d added 11 commits June 1, 2024 19:23

proxy bundling & launching logic

756fb9a

package-lock.json creation

627d429

certificates path

e955348

refactor get certificates path

625e303

get certificate spki utility

2f6d51c

launch proxy from renderer

fa6e16a

stop proxy event

b32ba3e

sending of proxy data

b5df1bf

launching & stopping browser functionality

ede1647

remove package-lock.json

34a5f9b

proxy binary for other os/arch

bc34546

Llandy3d commented Jun 3, 2024

View reviewed changes

Llandy3d requested review from going-confetti and e-fisher June 3, 2024 14:42

Llandy3d added 3 commits June 3, 2024 19:03

add @puppeteer/browsers dependency

a1f8c9d

readline for proxy stdout

82a84d5

base64 encode request/response content

a32af57

Llandy3d commented Jun 3, 2024

View reviewed changes

e-fisher approved these changes Jun 4, 2024

View reviewed changes

going-confetti reviewed Jun 4, 2024

View reviewed changes

Llandy3d commented Jun 4, 2024

View reviewed changes

Llandy3d added 2 commits June 4, 2024 11:33

fixes

052f68e

snake_case -> camelCase

75da8f3

going-confetti approved these changes Jun 4, 2024

View reviewed changes

Llandy3d merged commit d0389a2 into main Jun 4, 2024

Llandy3d deleted the proxy_porting branch June 4, 2024 10:18

github-actions bot mentioned this pull request Jan 14, 2025

chore(main): release 0.11.0 #405

Closed

croco-bot mentioned this pull request Jan 24, 2025

chore(main): release 0.13.0 #432

Merged

croco-bot mentioned this pull request Feb 13, 2025

chore(main): release 0.14.0 #489

Draft

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: proxy & browser capabilities #2

feat: proxy & browser capabilities #2

Llandy3d commented Jun 3, 2024

Llandy3d Jun 3, 2024

Llandy3d Jun 3, 2024

Llandy3d Jun 3, 2024

Llandy3d Jun 3, 2024

e-fisher left a comment

going-confetti left a comment

going-confetti Jun 4, 2024

going-confetti Jun 4, 2024

Llandy3d Jun 4, 2024

going-confetti Jun 4, 2024

going-confetti Jun 4, 2024

Llandy3d Jun 4, 2024

going-confetti Jun 4, 2024

Llandy3d Jun 4, 2024

Llandy3d Jun 4, 2024

going-confetti left a comment

feat: proxy & browser capabilities #2

feat: proxy & browser capabilities #2

Conversation

Llandy3d commented Jun 3, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

e-fisher left a comment

Choose a reason for hiding this comment

going-confetti left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

going-confetti left a comment

Choose a reason for hiding this comment