Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Source URI Reloading backports #35327

Merged
merged 7 commits into from
May 5, 2023

Conversation

michalpristas
Copy link
Contributor

@michalpristas michalpristas commented May 4, 2023

Backport of multiple PRs related to config/source uri reloading

elastic/elastic-agent#686
elastic/elastic-agent#1252

How to test:

Standalone:

needs to fail with proper message

Managed

  • install
  • update policy to have custom source uri
  • init upgrade
  • see it fail with host not resolved

Succ scenario:
change uri to http://localhost:8080 and start fileserver there
python3 -m http.server
directory needs to contain target asc, sha512 and archive files in location beats/elastic-agent

@michalpristas michalpristas self-assigned this May 4, 2023
@michalpristas michalpristas requested review from a team as code owners May 4, 2023 09:14
@michalpristas michalpristas requested review from AndersonQ, michel-laterman, rdner and fearful-symmetry and removed request for a team May 4, 2023 09:14
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label May 4, 2023
@botelastic
Copy link

botelastic bot commented May 4, 2023

This pull request doesn't have a Team:<team> label.

@elasticmachine
Copy link
Collaborator

elasticmachine commented May 4, 2023

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2023-05-05T07:04:24.626+0000

  • Duration: 129 min 47 sec

Test stats 🧪

Test Results
Failed 0
Passed 6267
Skipped 402
Total 6669

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

Copy link
Member

@pchila pchila left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple of code style nitpicks but the major issue I have with this PR are the 2 binary files added to metricbeat (I suppose it's a wrong commit somewhere?)

@@ -113,12 +115,89 @@ func NewOperator(

operator.initHandlerMap()

os.MkdirAll(config.DownloadConfig.TargetDirectory, 0755)
os.MkdirAll(config.DownloadConfig.InstallPath, 0755)
if err := os.MkdirAll(config.DownloadConfig.TargetDirectory, 0755); err != nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick because I know that this is dead (or soon to be) code most likely: can't we check if it's already exist instead of trying to create it every time and log a warning when we get error (which could be any kind of error), something like

Suggested change
if err := os.MkdirAll(config.DownloadConfig.TargetDirectory, 0755); err != nil {
_, err := os.Stat(config.DownloadConfig.TargetDirectory
if errors.Is(err, fs.ErrNotExist) {
err := os.MkdirAll(config.DownloadConfig.InstallPath, 0755)
if err != nil {
return nil, fmt.Errorf("failed creating %q: %v", config.DownloadConfig.TargetDirectory, err)
}
}
if err != nil {
return nil, fmt.Errorf("error accessing download target directory %q", config.DownloadConfig.TargetDirectory)
}

I realize it's a lot more verbose but the error checking should be more precise

Disclaimer: I wrote the code directly in this comment so it probably does not compile, please do not apply the suggestion as-is 😅

@@ -166,7 +245,7 @@ func (o *Operator) HandleConfig(cfg configrequest.Request) (err error) {

_, stateID, steps, ack, err := o.stateResolver.Resolve(cfg)
if err != nil {
if err == filterContextCancelled(err) {
if errors.Is(err, filterContextCancelled(err)) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

if tc.expectedTLS {
require.NotNil(t, cfg.TLS)
require.Equal(t, tc.expectedTLSEnabled, *cfg.TLS.Enabled)
//require.Equal(t, tc.expectedFingerprint, cfg.TLS.CATrustedFingerprint)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

leftover ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no it's backport, i'm copying code, this is just something not backported yet

Comment on lines +73 to +80
client, err := c.HTTPTransportSettings.Client(
httpcommon.WithAPMHTTPInstrumentation(),
)
if err != nil {
return errors.New(err, "http.downloader: failed to generate client out of config")
}

client.Transport = download.WithHeaders(client.Transport, download.Headers)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't we do it like in x-pack/elastic-agent/pkg/artifact/download/http/verifier.go below ?

Suggested change
client, err := c.HTTPTransportSettings.Client(
httpcommon.WithAPMHTTPInstrumentation(),
)
if err != nil {
return errors.New(err, "http.downloader: failed to generate client out of config")
}
client.Transport = download.WithHeaders(client.Transport, download.Headers)
client, err := c.HTTPTransportSettings.Client(
httpcommon.WithAPMHTTPInstrumentation(),
httpcommon.WithModRoundtripper(func(rt http.RoundTripper) http.RoundTripper {
return download.WithHeaders(rt, download.Headers)
}),
)
if err != nil {
return errors.New(err, "http.downloader: failed to generate client out of config")
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this binary file needed?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this binary file needed?

@michalpristas
Copy link
Contributor Author

@pchila i will ignore code style comments, as this is a code copy it would not be friendly for future backports.
those binary files i dont know how they got there, removing

@michalpristas michalpristas merged commit f9ee68f into elastic:7.17 May 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport needs_team Indicates that the issue/PR needs a Team:* label
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants