Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ability to custom handle redirects #5678

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions api/internal/loader/errors.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,12 @@ var (
ErrHTTP = errors.Errorf("HTTP Error")
ErrRtNotDir = errors.Errorf("must build at directory")
)

type RedirectionError struct {
NewPath string
Msg string
}

func (e *RedirectionError) Error() string {
return e.Msg
}
19 changes: 15 additions & 4 deletions api/internal/loader/fileloader.go
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,9 @@ type FileLoader struct {
cleaner func() error
}

// This redirect code does not process automaticaly by http client and we can process it manually
const MultipleChoicesRedirectCode = 300

// Repo returns the absolute path to the repo that contains Root if this fileLoader was created from a url
// or the empty string otherwise.
func (fl *FileLoader) Repo() string {
Expand Down Expand Up @@ -318,11 +321,19 @@ func (fl *FileLoader) httpClientGetContent(path string) ([]byte, error) {
defer resp.Body.Close()
// response unsuccessful
if resp.StatusCode < 200 || resp.StatusCode > 299 {
_, err = git.NewRepoSpecFromURL(path)
if err == nil {
return nil, errors.Errorf("URL is a git repository")
if resp.StatusCode == MultipleChoicesRedirectCode {
var newPath string = resp.Header.Get("Location")
return nil, &RedirectionError{
Msg: "Response is redirect",
NewPath: newPath,
}
Comment on lines +325 to +329
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe That is better to solve redirects here than return RedirectionError and a new path.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi! It does not work in all cases. For example if I return redirect to git. In that case httpClient has noe credentials for download content

} else {
_, err = git.NewRepoSpecFromURL(path)
if err == nil {
return nil, errors.Errorf("URL is a git repository")
}
return nil, fmt.Errorf("%w: status code %d (%s)", ErrHTTP, resp.StatusCode, http.StatusText(resp.StatusCode))
}
return nil, fmt.Errorf("%w: status code %d (%s)", ErrHTTP, resp.StatusCode, http.StatusText(resp.StatusCode))
}
content, err := io.ReadAll(resp.Body)
return content, errors.Wrap(err)
Expand Down
30 changes: 30 additions & 0 deletions api/internal/loader/fileloader_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ import (
"sigs.k8s.io/kustomize/api/ifc"
"sigs.k8s.io/kustomize/api/internal/git"
"sigs.k8s.io/kustomize/api/konfig"
"sigs.k8s.io/kustomize/kyaml/errors"
"sigs.k8s.io/kustomize/kyaml/filesys"
)

Expand Down Expand Up @@ -659,6 +660,35 @@ func TestLoaderHTTP(t *testing.T) {
_, err := l2.Load(x.path)
require.Error(err)
}

var testCaseRedirect = []testData{
{
path: "https://example.com/resource.yaml",
expectedContent: "https content",
},
Comment on lines +664 to +668
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have no plans to add another tests.

}
for _, x := range testCaseRedirect {
expectedLocation := "https://test.com/resource.yaml"
hc := makeFakeHTTPClient(func(req *http.Request) *http.Response {
response := &http.Response{
StatusCode: MultipleChoicesRedirectCode,
Body: io.NopCloser(bytes.NewBufferString("")),
Header: make(http.Header),
}
response.Header.Add("Location", expectedLocation)
return response
})
l2 := l1
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does l1 and l2 mean?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is loaders. I use solution from testcase from

l2.http = hc
_, err := l2.Load(x.path)
require.Error(err)
var redErr *RedirectionError
var path string = ""
if errors.As(err, &redErr) {
path = redErr.NewPath
}
require.Equal(expectedLocation, path)
}
}

// setupOnDisk sets up a file system on disk and directory that is cleaned after
Expand Down
5 changes: 5 additions & 0 deletions api/internal/localizer/localizer.go
Original file line number Diff line number Diff line change
Expand Up @@ -335,6 +335,11 @@ func (lc *localizer) localizeResource(path string) (string, error) {
}
}
if fileErr != nil {
var redErr *loader.RedirectionError
if errors.As(fileErr, &redErr) {
path = redErr.NewPath
}

var rootErr error
locPath, rootErr = lc.localizeRoot(path)
if rootErr != nil {
Expand Down
5 changes: 5 additions & 0 deletions api/internal/target/kusttarget.go
Original file line number Diff line number Diff line change
Expand Up @@ -423,7 +423,12 @@ func (kt *KustTarget) accumulateResources(
if errors.Is(errF, load.ErrHTTP) {
return nil, errF
}
var redErr *load.RedirectionError
if errors.As(errF, &redErr) {
path = redErr.NewPath
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this code is using redirect Location path directly.
I am worried that newPath will be used by no check.
like the redirected path requires redirecting more, or the redirected path is missing contents.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I investigated in these scenarious

  1. new loader works only with directory and git repo because of that all checks will be done in common way in
    func (fl *FileLoader) New(path string) (ifc.Loader, error) {
  2. If newPath will be a simple url in new loader it will be treated as directory and kustomize finish work with error in common way
  3. If some redirects (301 to 309 http codes) will happen in loading process
    resp, err := hc.Get(path)
    They will be processed by http client out of the box
    Following this points I think that all possible checks have been done

}
ldr, err := kt.ldr.New(path)

if err != nil {
// If accumulateFile found malformed YAML and there was a failure
// loading the resource as a base, then the resource is likely a
Expand Down
Loading