-
Notifications
You must be signed in to change notification settings - Fork 8.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Introduce external url service #81234
Introduce external url service #81234
Conversation
ebfffb1
to
7df77c2
Compare
@streamich & @elastic/kibana-platform I'd like your input on the approach I've taken here before getting this ready for review. |
window.location.assign( | ||
basePath.prepend(`/internal/external_redirect/goto?next=${encodeURIComponent(path)}`) | ||
); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
-
Where is this route defined? Couldn't find it in the PR changes, did I miss it?
-
I see we have a
ExternalUrlService
client-side service, why using a redirection instead of performing the validation directly on the client-side? -
This doesn't really protect against 'plain' links (
<a>
tags that could be displayed into markdown for example), and will force teams to manually implement the check via the API (when not usingnavigateToUrl
at least). However, I guess this is kinda the best we can do without some ugly top-level global handler that would probably causes the same issue we encountered when trying to implement it for the cross-app link navigation.
Overall, I'm alright with the approach.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Where is this route defined? Couldn't find it in the PR changes, did I miss it?
Ah, this was a holdover from a previous implementation. There should be no changes to this file.
This doesn't really protect against 'plain' links ( tags that could be displayed into markdown for example), and will force teams to manually implement the check via the API (when not using navigateToUrl at least). However, I guess this is kinda the best we can do without some ugly top-level global handler that would probably causes the same issue we encountered when trying to implement it for the cross-app link navigation.
That's true, and my intent was not to add any sort of protection in this PR. My goal was to provide the service that other teams could consume, in whatever way makes the most sense for them. Solving all of these use cases would blow up the scope of this work, and isn't something we've honestly allocated time for.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, this was a holdover from a previous implementation. There should be no changes to this file.
My goal was to provide the service that other teams could consume
Ok, so the PR is supposed to just expose this service from core
, without wiring it automatically anywhere, right? In that case, it sounds good to me, the implementation looks fine.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, so the PR is supposed to just expose this service from core, without wiring it automatically anywhere, right?
Yup, that's exactly right!
7df77c2
to
cbee187
Compare
Pinging @elastic/kibana-app-arch (Team:AppArch) |
Pinging @elastic/kibana-security (Team:Security) |
To be clear: you're proposing that, for example, Canvas alter its markdown element to check any link against this service? And then Dashboard checks its markdown panels' links using this service? If that is the case, we are guaranteed to have inconsistent implementations... or worse, missing implementations... where an administrator assumes external links are vetted across Kibana, when they aren't. Doesn't it make more sense to add a redirect/external url JS check in the window of Kibana that checks against this service, at a minimum? Alternatively, we should have a utility JS function and React component that can easily be taken up by plugins. We should also find ways to audit I guess I'm saying we should reverse the process here: write the utility function and, once all plugins have utilized it, then introduce this service to the codebase. But that's just IMHO... I may be overly cautious. :-) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Telemetry changes LGTM!
Only test snapshot updated to add the new externalUrl.validateUrl
function
@clintandrewhall thanks for the feedback -
For this specific example: I wouldn't have thought that Dashboard and Canvas would have separate implementations for markdown rendering. Is there a common markdown rendering service that we could use to handle this logic in a consistent way across Kibana?
We can certainly explore this further. One caveat is that this won't work for links that are "middle clicked", or otherwise asked to open in a new tab. I feel like the expectation for external links is to open in new tabs, so you don't lose your context, so I'm not sure how many scenarios this will catch on its own. There might be other complications that I'm not considering here - @pgayvallet mentioned that they encountered issues when implementing the global app link redirect service.
Can you expand on this a bit? The goal of this PR was to introduce a utility function that could be easily taken up by plugins, so I think I'm misunderstanding something.
You're feedback is more than welcome, thanks for the input! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall looking good. A few NITS and questions
); | ||
|
||
export const config = { | ||
path: 'externalUrl', |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The eternal question of externalUrl
vs external_url
prefix. @elastic/kibana-platform, which one do we prefer here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
according to #52284:
New public APIs should respect snake_case conventions on a best effort basis
config is public API too
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
According to #7444:
Until we've refactored all config keys in a single big refactor, all existing and new config keys should keep using camelCase for consistency.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@joshdover it seems we need to update your comment in #7444. We do the automatic conversion to snake_case
if a plugin config path is not provided
configPath: manifest.configPath || snakeCase(manifest.id), |
constructor(rawHttpConfig: HttpConfigType, rawCspConfig: CspConfigType) { | ||
constructor( | ||
rawHttpConfig: HttpConfigType, | ||
rawCspConfig: CspConfigType, | ||
rawExternalUrlConfig: ExternalUrlConfig | ||
) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do we need to put this externalUrl
config into the http
one again? It's only used internally by the rendering
service, right? It's to avoid creating a new service just to expose the config from it, right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why don't we declare it as a separate service? Should we make the external_url
service space-aware it introduces http
dependency on SO / UiSettings. Which we must avoid
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do we need to put this externalUrl config into the http one again? It's only used internally by the rendering service, right? It's to avoid creating a new service just to expose the config from it, right?
Yeah I was trying to take the path of least resistance here. It seemed like a logical place to put this service.
Why don't we declare it as a separate service?
We can if you'd prefer.
Should we make the external_url service space-aware it introduces http dependency on SO / UiSettings. Which we must avoid
I didn't intend for this to be space-aware, and I don't see a need to do so in the near term.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I didn't intend for this to be space-aware, and I don't see a need to do so in the near term.
ok, I thought it's going to be a user-specific setting in the future, but I cannot find any references to such a requirement in the parent issue.
Yea. there are issues regarding handlers execution order between handlers bound from the react world and global 'vanilla' handlers. The global handler (which would be the 'external url' handler) may be executed first (even if bound on an upper element such as document or body), resulting in the lower-level, react handler to never execute, or to not be able to prevent the default. Also, a global handler does not handle the common scenario of links navigating via onClick (but this example would not work anyway due to previously mentioned limitation)
And yea, as @legrego already mentioned, that would not work with |
// True => allow request, don't check other rules | ||
// False => reject request, don't check other rules | ||
// Undefined => Not yet known, proceed to next rule | ||
const allowed = rules.reduce((result: boolean | undefined, rule) => { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
optional: IMO, with a loop you don't need this comment:
for (const rule of rules) {
const hostMatch = rule.host ? isHostMatch(url.hostname || '', rule.host) : true;
const protocolMatch = rule.protocol ? isProtocolMatch(url.protocol, rule.protocol) : true;
const isRuleMatch = hostMatch && protocolMatch;
if (isRuleMatch) {
return rule.allow ? url : null;
}
}
return null;
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, I can simplify this. I borrowed this logic from the reporting implementation, but we likely don't have to do things the same way here.
); | ||
|
||
export const config = { | ||
path: 'externalUrl', |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
according to #52284:
New public APIs should respect snake_case conventions on a best effort basis
config is public API too
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This doesn't really protect against 'plain' links ( tags that could be displayed into markdown for example)
it falls into our scope of responsibility. we need to create an issue to integrate this service for all the links.
Yea. there are issues regarding handlers execution order between handlers bound from the react world and global 'vanilla' handlers. The global handler (which would be the 'external url' handler) may be executed first (even if bound on an upper element such as document or body), resulting in the lower-level, react handler to never execute, or to not be able to prevent the default.
Hasn't it been fixed in React v17 https://github.com/facebook/react/blob/master/CHANGELOG.md#1700-october-20-2020 ?
Do we have a code snippet to test?
constructor(rawHttpConfig: HttpConfigType, rawCspConfig: CspConfigType) { | ||
constructor( | ||
rawHttpConfig: HttpConfigType, | ||
rawCspConfig: CspConfigType, | ||
rawExternalUrlConfig: ExternalUrlConfig | ||
) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why don't we declare it as a separate service? Should we make the external_url
service space-aware it introduces http
dependency on SO / UiSettings. Which we must avoid
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code LGTM.
// Accumulator has three potential values here: | ||
// True => allow request, don't check other rules | ||
// False => reject request, don't check other rules | ||
// Undefined => Not yet known, proceed to next rule |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What do you think about making rejection more powerful, i.e. if there is true
we still check all other rules for explicit denial?
// Accumulator has three potential values here: | |
// True => allow request, don't check other rules | |
// False => reject request, don't check other rules | |
// Undefined => Not yet known, proceed to next rule | |
// Accumulator has three potential values here: | |
// True => will allow request, if there are no denial in remaining rules | |
// False => reject request, don't check other rules | |
// Undefined => Not yet known, proceed to next rule |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1.
I understand the desire to stay aligned with the aforementioned prior art, but I feel that this is safer.
Reviewing now |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Took a first pass, a few nits!
// Accumulator has three potential values here: | ||
// True => allow request, don't check other rules | ||
// False => reject request, don't check other rules | ||
// Undefined => Not yet known, proceed to next rule |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1.
I understand the desire to stay aligned with the aforementioned prior art, but I feel that this is safer.
@jportner @pgayvallet I've addressed your feedback if you're free for another review round. I have not split this out into a distinct service yet, but I can do so if you feel strongly enough about it. I also did not change the config key from |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changes so far look good! A few more comments.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. I created #85006 to discuss about the possible implementation / usage in core.
I have not split this out into a distinct service yet, but I can do so if you feel strongly enough about it.
It's fine not to do it. That can be done later if we find a reason to do so.
I also did not change the config key from externalUrl to external_url because we have conflicting guidance, so I aired on the side of consistency within Kibana for the time being.
Fine with me, unless @restrry or @joshdover have an objection about that.
const policySchema = schema.object({ | ||
allow: allowSchema, | ||
protocol: schema.maybe(protocolSchema), | ||
host: schema.maybe(hostSchema), | ||
}); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Don't we need the
validate: ({protocol, host}) => {
if(!host && !protocol) return 'must have either...'
}
part? or it this done elsewhere?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I intentionally removed this in order to support a default "allow all" policy (via #81234 (comment)).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think we needed to remove it to support default allow-all, as the default value of the schema isn't subject to validation.
That said, I don't feel strongly about this change either way.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd prefer to have our default value honor the validation schema. That feels safer and less error prone to me.
@elasticmachine merge upstream |
…ibana into core/external-redirect-service
function* getHostHashes(actualHost: string) { | ||
yield new Sha256().update(actualHost, 'utf8').digest('hex'); | ||
let host = actualHost.substr(actualHost.indexOf('.') + 1); | ||
while (host) { | ||
yield new Sha256().update(host, 'utf8').digest('hex'); | ||
if (host.indexOf('.') === -1) { | ||
break; | ||
} | ||
host = host.substr(host.indexOf('.') + 1); | ||
} | ||
} | ||
|
||
const isHostMatch = (actualHost: string, ruleHostHash: string) => { | ||
for (const hash of getHostHashes(actualHost)) { | ||
if (hash === ruleHostHash) { | ||
return true; | ||
} | ||
} | ||
return false; | ||
}; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like what you did here and it's really elegant.
Nit: There's an edge case that this doesn't work for. A URI's host
component can be 1. an IP literal encapsulated within square brackets, 2. an IPv4 address in dotted-decimal form, or 3. a registered name (RFC 3986). The RFC says:
A registered name intended for lookup in the DNS uses the syntax
defined in Section 3.5 of [RFC1034] and Section 2.1 of [RFC1123].
Such a name consists of a sequence of domain labels separated by ".",
each domain label starting and ending with an alphanumeric character
and possibly also containing "-" characters. The rightmost domain
label of a fully qualified domain name in DNS may be followed by a
single "." and should be if it is necessary to distinguish between
the complete domain name and some local domain.
In other words: google.com
is shorthand for google.com.
, and this appears to hold true for IPv4 addresses as well (at least, from my local testing in a terminal 🙂).
I'm thinking this could account for the edge case without changing your approach:
Click for diff
diff --git a/src/core/public/http/external_url_service.test.ts b/src/core/public/http/external_url_service.test.ts
index d5371be253f..ccb6a5a55fc 100644
--- a/src/core/public/http/external_url_service.test.ts
+++ b/src/core/public/http/external_url_service.test.ts
@@ -33,10 +33,15 @@ const setupService = ({
serverBasePath: string;
policy: ExternalUrlConfig['policy'];
}) => {
- const hashedPolicies = policy.map((entry) => ({
- ...entry,
- host: entry.host ? new Sha256().update(entry.host, 'utf8').digest('hex') : undefined,
- }));
+ const hashedPolicies = policy.map((entry) => {
+ // If the host contains a period (i.e., it is a registered name or IPv4 address), append a `.` if it doesn't already contain one
+ const hostToHash =
+ entry?.host?.indexOf('.') && !entry.host.endsWith('.') ? `${entry.host}.` : entry.host;
+ return {
+ ...entry,
+ host: hostToHash ? new Sha256().update(hostToHash, 'utf8').digest('hex') : undefined,
+ };
+ });
const injectedMetadata = injectedMetadataServiceMock.createSetupContract();
injectedMetadata.getExternalUrlConfig.mockReturnValue({ policy: hashedPolicies });
injectedMetadata.getServerBasePath.mockReturnValue(serverBasePath);
@@ -332,6 +337,25 @@ describe('External Url Service', () => {
expect(result?.toString()).toEqual(urlCandidate);
});
+ it('allows external urls with a partially matching host and protocol in the allow list when the URL includes the root domain', () => {
+ const { setup } = setupService({
+ location,
+ serverBasePath,
+ policy: [
+ {
+ allow: true,
+ host: 'google.com',
+ protocol: 'https',
+ },
+ ],
+ });
+ const urlCandidate = `https://www.google.com./foo?bar=baz`;
+ const result = setup.validateUrl(urlCandidate);
+
+ expect(result).toBeInstanceOf(URL);
+ expect(result?.toString()).toEqual(urlCandidate);
+ });
+
it('allows external urls that specify a locally addressable host', () => {
const { setup } = setupService({
location,
diff --git a/src/core/public/http/external_url_service.ts b/src/core/public/http/external_url_service.ts
index 05ebe00e7de..7d720615413 100644
--- a/src/core/public/http/external_url_service.ts
+++ b/src/core/public/http/external_url_service.ts
@@ -42,7 +42,10 @@ function* getHostHashes(actualHost: string) {
}
const isHostMatch = (actualHost: string, ruleHostHash: string) => {
- for (const hash of getHostHashes(actualHost)) {
+ // If the host contains a period (i.e., it is a registered name or IPv4 address), append a `.` if it doesn't already contain one
+ const hostToHash =
+ actualHost.indexOf('.') && !actualHost.endsWith('.') ? `${actualHost}.` : actualHost;
+ for (const hash of getHostHashes(hostToHash)) {
if (hash === ruleHostHash) {
return true;
}
diff --git a/src/core/server/external_url/external_url_config.ts b/src/core/server/external_url/external_url_config.ts
index ad2a7940ec0..acd4a5f6112 100644
--- a/src/core/server/external_url/external_url_config.ts
+++ b/src/core/server/external_url/external_url_config.ts
@@ -85,9 +85,12 @@ export class ExternalUrlConfig implements IExternalUrlConfig {
constructor(rawConfig: IExternalUrlConfig) {
this.policy = rawConfig.policy.map((entry) => {
if (entry.host) {
+ // If the host contains a period (i.e., it is a registered name or IPv4 address), append a `.` if it doesn't already contain one
+ const hostToHash =
+ entry.host.indexOf('.') && !entry.host.endsWith('.') ? `${entry.host}.` : entry.host;
return {
...entry,
- host: createSHA256Hash(entry.host),
+ host: createSHA256Hash(hostToHash),
};
}
return entry;
What do you think?
Edit: this still wouldn't work for hostnames without a dot, like localhost
. So maybe the entry.host.indexOf('.')
check needs to be !entry.host.indexOf('[')
, as a square bracket cannot be in an IPv4 address or a registered name.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great find! Addressed in 85c1dfb
(#81234)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!!
💚 Build SucceededMetrics [docs]Module Count
Distributable file count
Page load bundle
History
To update your PR or re-run it, just comment with: |
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> # Conflicts: # src/core/public/index.ts
* master: (53 commits) Fixing recovered instance reference bug (elastic#85412) Switch to new elasticsearch client for Visualizations (elastic#85245) Switch to new elasticsearch client for TSVB (elastic#85275) Switch to new elasticsearch client for Vega (elastic#85280) [ILM] Add shrink field to hot phase (elastic#84087) Add rolling-file appender to core logging (elastic#84735) [APM] Service overview: Dependencies table (elastic#83416) [Uptime ]Update empty message for certs list (elastic#78575) [Graph] Fix graph saved object references (elastic#85295) [APM] Create new API's to return Latency and Throughput charts (elastic#85242) [Advanced settings] Reset to default for empty strings (elastic#85137) [SECURITY SOLUTION] Bundles _source -> Fields + able to sort on multiple fields in Timeline (elastic#83761) [Fleet] Update agent listing for better status reporting (elastic#84798) [APM] enable 'sanitize_field_names' for Go (elastic#85373) Update dependency @elastic/charts to v24.4.0 (elastic#85452) Introduce external url service (elastic#81234) Deprecate disabling the security plugin (elastic#85159) [FLEET] New Integration Policy Details page for use in Integrations section (elastic#85355) [Security Solutions][Detection Engine] Fixes one liner access control with find_rules REST API chore: 🤖 remove extraPublicDirs (elastic#85454) ...
Summary
👀 Important
This PR defines a service for determining of an external destination is allowed, but it makes no effort to prevent Kibana from sending users to this location. Given the breadth of Kibana's features, I feel that is an initiative that each team should undertake for their own solutions once this capability is introduced into core.
For this reason, I think we should either:
Plugin API Changes
Introduces a new
externalUrl.policy
configuration option, which allows administrators to decide if Kibana should "allow" a user to follow a link to a location outside of Kibana:Multiple policy definitions can be supplied. Each definition specifies some combination of host and protocol, and whether that combination should be allowed or denied. The list of policies are checked in order to determine if access should be granted:
This example policy yields the following result:
https://google.com
http://google.com
google.com
overhttp
https://danger.google.com
danger
subdomain is explicitly forbiddenftp://www.elastic.co
scp://some.website.com
https://other-site.com
other-site.com
over any protocol.To test if a URL is allowed by the policy, front-end plugins can use the
core.http.externalUrl.validateUrl
function available on thesetup
contract:This function returns a WHATWG compliant URL if the requested destination is allowed via the external url policy, and
null
otherwise.Prior Art
In an effort to maintain relative consistency, the policy definitions defined here closely resemble that of
reporting.networkPolicy
, as described in their docs.Resolves #69411
Checklist
Delete any items that are not applicable to this PR.
For maintainers