Allow custom robots text pages #1104
Conversation
pkg/app/pagewriter/static_pages.go
Outdated
| _, err := rw.Write(content) | ||
| if err != nil { | ||
| logger.Printf("Error writing %q: %v", pageName, err) | ||
| s.errorPageWriter.WriteErrorPage(rw, http.StatusInternalServerError, "", err.Error()) |
There was a problem hiding this comment.
This will need a req *http.Request after my RequestID PR to extract out the {{.RequestID}} from the request.
pkg/app/pagewriter/static_pages.go
Outdated
| // Statis files include: | ||
| // - robots.txt | ||
| func loadStaticPages(customDir string) (map[string][]byte, error) { | ||
| pages := make(map[string][]byte) |
There was a problem hiding this comment.
For how this is passed around and manipulated by reference in all these functions, does this make sense as a type (or full struct) with these all as member methods manipulating it?
There was a problem hiding this comment.
maybe a struct it looks like with the pages & customDir as member fields?
| Expect(err).ToNot(HaveOccurred()) | ||
|
|
||
| robotsTxtFile := filepath.Join(customDir, robotsTxtName) | ||
| Expect(ioutil.WriteFile(robotsTxtFile, []byte(customRobots), 0600)).To(Succeed()) |
There was a problem hiding this comment.
| Expect(ioutil.WriteFile(robotsTxtFile, []byte(customRobots), 0600)).To(Succeed()) | |
| Expect(ioutil.WriteFile(robotsTxtFile, []byte(customRobots), 0400)).To(Succeed()) |
|
|
||
| var _ = Describe("Static Pages", func() { | ||
| var customDir string | ||
| const customRobots = "I AM A ROBOT!!!" |
There was a problem hiding this comment.
NIT: Can we have this custom test payload resemble the robots.txt traditional structure?
|
@NickMeves please take a look now it has been updated, created a separate pageGetter struct, not sure whether it makes it simpler or not, WDYT? |
|
Added changelog entry, not other changes since last review |
Description
This allows users to replace our static robots.txt content by using the templates dir option and placing a robots text inside the directory.
Motivation and Context
Users should be able to customise this content.
Fixes #1068
This also sets the stage for serving other static files.
At the moment, we embed CSS and other files in our pages, maybe we went to serve that as static files too?
How Has This Been Tested?
Unit testing only.
Checklist: