[Refactor] Replace url parse format resolve with whatwg url #2910

wanglam · 2022-11-22T08:46:26Z

Description

This PR mainly replaces the url.parse/url.format/url.resolve function in the code base with whatwg-url. The main change is to change the url.parse to new URL(), and change url.format to [URL object].toString and change the url.resolve to new URL() toString(). According to the discussion in #1561 , this PR only deals with the parts that can be replaced quickly. Therefore, most of the modifications only involve simple function replacement. Another major change is to replace the url parts in test config with URL object for transmission. Most files are modified with a file based commit, so we could just delete this commit if this change isn't necessary.

Issues Resolved

#1561

Check List

codecov-commenter · 2022-11-22T11:30:24Z

Codecov Report

Merging #2910 (a846ba1) into main (de06344) will decrease coverage by 0.07%.
The diff coverage is 48.00%.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

@@            Coverage Diff             @@
##             main    #2910      +/-   ##
==========================================
- Coverage   66.45%   66.38%   -0.07%     
==========================================
  Files        3208     3208              
  Lines       61593    61616      +23     
  Branches     9502     9504       +2     
==========================================
- Hits        40932    40906      -26     
- Misses      18384    18425      +41     
- Partials     2277     2285       +8

Flag	Coverage Δ
Linux	`66.38% <48.00%> (-0.02%)`	⬇️
Windows	`?`

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...s/osd-test/src/failed_tests_reporter/github_api.ts	`7.84% <ø> (-1.78%)`	⬇️
...st/src/functional_test_runner/lib/config/config.ts	`43.90% <0.00%> (-2.05%)`	⬇️
...st/src/functional_test_runner/lib/config/schema.ts	`94.73% <ø> (ø)`
...unner/lib/docker_servers/docker_servers_service.ts	`27.94% <ø> (-1.05%)`	⬇️
...egacy_opensearch/legacy_opensearch_test_cluster.js	`20.45% <0.00%> (-1.77%)`	⬇️
...st/src/legacy_opensearch/opensearch_test_config.js	`15.78% <0.00%> (-10.88%)`	⬇️
packages/osd-test/src/osd/osd_test_config.ts	`10.00% <0.00%> (-5.79%)`	⬇️
src/core/public/chrome/ui/header/header_logo.tsx	`34.09% <0.00%> (ø)`
src/core/public/chrome/ui/header/home_loader.tsx	`17.14% <0.00%> (ø)`
src/core/server/http/base_path_proxy_server.ts	`6.66% <0.00%> (-0.32%)`	⬇️
... and 12 more

... and 7 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

…matUrl Signed-off-by: Lin Wang <wonglam@amazon.com>

Signed-off-by: Lin Wang <wonglam@amazon.com>

…s/docker_servers_service.ts Signed-off-by: Lin Wang <wonglam@amazon.com>

Signed-off-by: Lin Wang <wonglam@amazon.com>

…dler.ts Signed-off-by: Lin Wang <wonglam@amazon.com>

Signed-off-by: Lin Wang <wonglam@amazon.com>

…app_links.ts Signed-off-by: Lin Wang <wonglam@amazon.com>

Signed-off-by: Lin Wang <wonglam@amazon.com>

…th-whatwg-url

…context_page.ts Signed-off-by: Lin Wang <wonglam@amazon.com>

Signed-off-by: Lin Wang <wonglam@amazon.com>

…th-whatwg-url

Signed-off-by: Lin Wang <wonglam@amazon.com>

…th-whatwg-url

Signed-off-by: Lin Wang <wonglam@amazon.com>

wanglam · 2023-02-14T19:30:27Z

@joshuarrrr @AMoo-Miki I have pushed all the file changes. Feel free to help me review again. Thanks.

joshuarrrr · 2023-02-15T19:57:45Z

Thanks! We'll try to see if we can find the capacity to review this week.

joshuarrrr

@wanglam Looking good?

A couple minor questions. The only major request I have left is to centralize the .toString().slice(0, -1) manipulation so that it's not peppered throughout the code.

joshuarrrr · 2023-02-17T00:25:53Z

packages/osd-opensearch-archiver/src/cli.ts

+        opensearchDashboardsUrl = config
+          .get('servers.opensearchDashboards')
+          .fullURL.toString()
+          .slice(0, -1) as string;


do we still need the as string?

Yeah. The config.get('servers.opensearchDashboards') will return an any object. The TS complier won't know its type. So I add the as string.

packages/osd-test/src/osd/osd_test_config.ts

test/server_integration/http/platform/headers.ts

…th-whatwg-url

Signed-off-by: Lin Wang <wonglam@amazon.com>

packages/osd-test/src/legacy_opensearch/opensearch_test_config.js

joshuarrrr

@wanglam Thanks for the perseverance! LGTM!

packages/osd-test/src/osd/osd_test_config.ts

AMoo-Miki

There are some leftovers of the legacy URL APIs but we could deal with them incrementally after merging this since these changes are slated for v3.0.0.

joshuarrrr · 2023-03-24T18:13:04Z

Yeah, here's the scope of this PR: #1561 (comment)

After merging, I'll create a follow-up issue for the trickier bits.

…ch-project#2910) Replace the url.parse/url.format/url.resolve function in the code base with whatwg-url. The main change is to change the url.parse to new URL(), and change url.format to [URL object].toString and change the url.resolve to new URL() toString(). Another major change is to replace the url parts in test config with URL object for transmission. Partially completes opensearch-project#1561 Signed-off-by: Lin Wang <wonglam@amazon.com> Co-authored-by: Josh Romero <rmerqg@amazon.com>

…ch-project#2910) Replace the url.parse/url.format/url.resolve function in the code base with whatwg-url. The main change is to change the url.parse to new URL(), and change url.format to [URL object].toString and change the url.resolve to new URL() toString(). Another major change is to replace the url parts in test config with URL object for transmission. Partially completes opensearch-project#1561 Signed-off-by: Lin Wang <wonglam@amazon.com> Co-authored-by: Josh Romero <rmerqg@amazon.com> Signed-off-by: David Sinclair <david@sinclair.tech>

wanglam mentioned this pull request Nov 22, 2022

[Chore] Update deprecated url methods (url.parse(), url.format()) #1561

Open

wanglam marked this pull request as ready for review November 22, 2022 09:53

wanglam requested a review from a team as a code owner November 22, 2022 09:53

wanglam added 25 commits November 23, 2022 10:43

update test/server_integration/http/ssl_redirect/config.js Url to for…

4a67aec

…matUrl Signed-off-by: Lin Wang <wonglam@amazon.com>

update for examples/url_generators_examples/public/url_generator.ts

c37c80d

Signed-off-by: Lin Wang <wonglam@amazon.com>

update packages/osd-dev-utils/src/osd_client/osd_client_requester.ts

c2511a8

Signed-off-by: Lin Wang <wonglam@amazon.com>

update src/cli_plugin/install/download.js

2921fda

Signed-off-by: Lin Wang <wonglam@amazon.com>

update packages/osd-test/src/failed_tests_reporter/github_api.ts

3247937

Signed-off-by: Lin Wang <wonglam@amazon.com>

update packages/osd-test/src/functional_test_runner/lib/docker_server…

defe504

…s/docker_servers_service.ts Signed-off-by: Lin Wang <wonglam@amazon.com>

update src/cli/serve/serve.js

12cc53f

Signed-off-by: Lin Wang <wonglam@amazon.com>

update src/core/public/application/utils/parse_app_url.ts

2218ede

Signed-off-by: Lin Wang <wonglam@amazon.com>

update src/core/public/chrome/chrome_service.tsx

3fc62a3

Signed-off-by: Lin Wang <wonglam@amazon.com>

update src/core/public/chrome/ui/header/header_logo.tsx

d1cc130

Signed-off-by: Lin Wang <wonglam@amazon.com>

update src/core/server/http/base_path_proxy_server.ts

14b5497

Signed-off-by: Lin Wang <wonglam@amazon.com>

update src/core/server/http/http_server.mocks.ts

a76fd60

Signed-off-by: Lin Wang <wonglam@amazon.com>

update src/core/server/http/http_server.ts

592301b

Signed-off-by: Lin Wang <wonglam@amazon.com>

update src/core/server/http/https_redirect_server.ts

5fabfd3

Signed-off-by: Lin Wang <wonglam@amazon.com>

update src/core/server/opensearch/legacy/opensearch_client_config.ts

7c5e014

Signed-off-by: Lin Wang <wonglam@amazon.com>

update src/plugins/console/server/lib/opensearch_proxy_config.ts

1517b5a

Signed-off-by: Lin Wang <wonglam@amazon.com>

update src/plugins/console/server/lib/proxy_config.ts

7248b3f

Signed-off-by: Lin Wang <wonglam@amazon.com>

update src/plugins/console/server/routes/api/console/proxy/create_han…

fea59fe

…dler.ts Signed-off-by: Lin Wang <wonglam@amazon.com>

update src/plugins/maps_legacy/public/map/service_settings.test.js

73a0cb4

Signed-off-by: Lin Wang <wonglam@amazon.com>

update src/plugins/share/public/lib/url_shortener.ts

0a293d1

Signed-off-by: Lin Wang <wonglam@amazon.com>

update test/plugin_functional/test_suites/application_links/redirect_…

7792efa

…app_links.ts Signed-off-by: Lin Wang <wonglam@amazon.com>

update getOpenSearchDashboardsUrl method in funtional tests

652a81c

Signed-off-by: Lin Wang <wonglam@amazon.com>

change config servers.opensearch and servers.opensearchDashboards to URL

7b0a2fc

Signed-off-by: Lin Wang <wonglam@amazon.com>

update src/plugins/share/public/components/url_panel_content.tsx

90e29a1

Signed-off-by: Lin Wang <wonglam@amazon.com>

update expected url due to parentheses and single quotation mark encoded

9e8818c

Signed-off-by: Lin Wang <wonglam@amazon.com>

wanglam force-pushed the refactor-replace-url-parse-format-resolve-with-whatwg-url branch from 1eaf259 to 9e8818c Compare November 23, 2022 02:43

wanglam force-pushed the refactor-replace-url-parse-format-resolve-with-whatwg-url branch from 32c4581 to a2d7108 Compare February 9, 2023 12:50

wanglam added 12 commits February 9, 2023 21:07

fix: change getURLParts to getUrlParts

32ec422

Signed-off-by: Lin Wang <wonglam@amazon.com>

Merge branch 'main' into refactor-replace-url-parse-format-resolve-wi…

3408cc5

…th-whatwg-url

fix: url parts pass to getUrl.noAuth in test/functional/page_objects/…

27529b1

…context_page.ts Signed-off-by: Lin Wang <wonglam@amazon.com>

fix: colon suffix missed in protocol

803c96a

Signed-off-by: Lin Wang <wonglam@amazon.com>

fix: remove all slash suffix for fullURL toString

63a8e87

Signed-off-by: Lin Wang <wonglam@amazon.com>

fix: slice(0, 1) cause invalid URL

12d2e78

Signed-off-by: Lin Wang <wonglam@amazon.com>

Merge branch 'main' into refactor-replace-url-parse-format-resolve-wi…

4b061b3

…th-whatwg-url

fix: use pared protocol, hostname and port

ec8ce6a

Signed-off-by: Lin Wang <wonglam@amazon.com>

fix: clone new URL and modify username and password

d8ad24d

Signed-off-by: Lin Wang <wonglam@amazon.com>

fix: slice(0 - 1) cause invalid opensearch host

c74698b

Signed-off-by: Lin Wang <wonglam@amazon.com>

Merge branch 'main' into refactor-replace-url-parse-format-resolve-wi…

9154295

…th-whatwg-url

update src/core/public/chrome/ui/header/home_loader.tsx

6ebdd20

Signed-off-by: Lin Wang <wonglam@amazon.com>

joshuarrrr changed the title ~~[Refactor]Refactor replace url parse format resolve with whatwg url~~ [Refactor] Replace url parse format resolve with whatwg url Feb 16, 2023

ashwin-pc assigned joshuarrrr Mar 7, 2023

joshuarrrr reviewed Mar 20, 2023

View reviewed changes

wanglam added 2 commits March 21, 2023 15:48

Merge branch 'main' into refactor-replace-url-parse-format-resolve-wi…

3d93282

…th-whatwg-url

Add serverUrl property and refactor slice(0, -1) to it

a846ba1

Signed-off-by: Lin Wang <wonglam@amazon.com>

joshuarrrr reviewed Mar 22, 2023

View reviewed changes

packages/osd-test/src/legacy_opensearch/opensearch_test_config.js Show resolved Hide resolved

joshuarrrr approved these changes Mar 24, 2023

View reviewed changes

packages/osd-test/src/osd/osd_test_config.ts Show resolved Hide resolved

AMoo-Miki approved these changes Mar 24, 2023

View reviewed changes

joshuarrrr merged commit 2de11ff into opensearch-project:main Mar 24, 2023

joshuarrrr mentioned this pull request Apr 12, 2023

Major.minor.patch definitions opensearch-project/opensearch-dashboards-sdk-js#41

Open

This was referenced Jul 23, 2024

[Manual] [Backport 2.x] [Look&Feel] Replace browser tooltip usage with OUI tooltip #7366

Merged

[Manual] [Backport 2.16] [Look&Feel] Replace browser tooltip usage with OUI tooltip #7465

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Refactor] Replace url parse format resolve with whatwg url #2910

[Refactor] Replace url parse format resolve with whatwg url #2910

wanglam commented Nov 22, 2022 •

edited

Loading

codecov-commenter commented Nov 22, 2022 •

edited

Loading

wanglam commented Feb 14, 2023

joshuarrrr commented Feb 15, 2023

joshuarrrr left a comment

joshuarrrr Feb 17, 2023

wanglam Mar 21, 2023

joshuarrrr left a comment

AMoo-Miki left a comment

joshuarrrr commented Mar 24, 2023

[Refactor] Replace url parse format resolve with whatwg url #2910

[Refactor] Replace url parse format resolve with whatwg url #2910

Conversation

wanglam commented Nov 22, 2022 • edited Loading

Description

Issues Resolved

Check List

codecov-commenter commented Nov 22, 2022 • edited Loading

Codecov Report

wanglam commented Feb 14, 2023

joshuarrrr commented Feb 15, 2023

joshuarrrr left a comment

Choose a reason for hiding this comment

joshuarrrr Feb 17, 2023

Choose a reason for hiding this comment

wanglam Mar 21, 2023

Choose a reason for hiding this comment

joshuarrrr left a comment

Choose a reason for hiding this comment

AMoo-Miki left a comment

Choose a reason for hiding this comment

joshuarrrr commented Mar 24, 2023

wanglam commented Nov 22, 2022 •

edited

Loading

codecov-commenter commented Nov 22, 2022 •

edited

Loading