-
Notifications
You must be signed in to change notification settings - Fork 8.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Scope-able elasticsearch clients #39430
Comments
Pinging @elastic/kibana-platform |
Pinging @elastic/kibana-security |
I can't think of anything that doesn't end up going through the elasticsearch service that needs this ability.
As far as I know, they exist because we don't have API Keys for background tasks and we're essentially emulating this behavior. Once we start using api keys, we'll need some mechanism of supplying this api key when using the Elasticsearch client, similar to the "fake request". Renaming it to something like
Once we get API keys, they will be doing so. We aren't able to use the hacky solutions like what Reporting does now because of limitations with the token based auth providers. One other thing we might have to take into consideration is we currently support a |
Not much to add here from my side, just one note:
The It happened so that scope consists of mainly authentication information right now (+anything UA sent to Kibana and that's allowed by I mention this because I think that if we want to override just one header (or one specific part of the ^scope^) we can't just do something like this : asScoped({headers: {authorization}}).callAsCurrentUser() We'll need to preserve any other headers that could have been added to the ^scope^: asScoped({headers: {...originalRequest.headers, authorization }}).callAsCurrentUser() We're doing this in "authc in NP" PR right now and if we have just a few places where we redefine scope that way - not a big deal. But if this boilerplate code will be all over the place, I'd rather reconsider our decision to forbid scope-header overrides on a call basis, e.g. allow this call even if we override scopedClientThatCameFromNPHandler.callAsCurrentUser(...., { headers: {authorization }}) |
We didn't forbid them. elasticsearch-js has the opposite logic, but aligning our code with it means we introduce breaking changes. |
I meant we forbid overriding them (default headers, passed through
I don't think it'd be a breaking change, it seems we never backported #34610 to 7.x. |
#34610 is not related to breaking changes per se. |
In order to ease migration to the NP, I'd rather keep the current behavior of not allowing these headers to be overridden. We can change this behavior when we introduce the new elasticsearch-js client along side the legacy one in #35508. We plan to do this in 7.5. This breaks up the migration process for existing plugins so they can deal with this change (and any other breaking changes) all at once. @Bamieh I am curious if #34610 was not backported to 7.x intentionally. If this was intentional, I am worried about the inconsistency between master and 7.x. |
@joshdover it was not intentional. Backported here: #40429 |
When Security migrated to the NP they stumbled upon a requirement that
Also, SavedObject service doesn't specify possible types for a
|
I'm discovering this part of the code, so please tell me if I'm saying anything wrong.
I will not talk about 'legacy' request here, as they should become core internal implementation details post 8.0
This is only true with the legacy SO service. NP one expects a KibanaRequest (even if effectively working with other types if force-casted)
This one seems tricky with current implementation, as we are keeping reference of the original request inside kibana/src/core/server/http/router/request.ts Lines 175 to 176 in 3bdbcd0
We make the assumption that this is the actual original HAPI request, and relies on it on some places, such as
Also current implementation to retrieve the auth headers is based on the fact that the http server / auth hook keep track of them kibana/src/core/server/elasticsearch/cluster_client.ts Lines 249 to 259 in 56041f0
(
As the behavior between actual requests 'received' by the http server and any other kind of requests differ (see the Maybe interface AuthorizationContext {
headers: Headers;
} and then adapt the ES/SO(UI?) APIs to accept Another option would be to have the 'request' be the single source of truth regarding authz headers. This would mean getting rid of the I would use a slightly different interface in that case: interface AuthorizationContext {
getAuthorizationHeaders: () => Headers;
} .This way, we could have ES/SO APIs only accept the new
We would have a fallback mechanism for legacy until 8.0, by checking the presence of |
@kobelb It appears that Alerting is now using API keys. Is there any reason we can't or shouldn't change the Upgrade Assistant and Reporting code to use API keys as well?
From what I understand the original reason for having the indirect way of getting auth headers was so that we could hide all of that information from plugin code. We have had at least one security vulnerability in the past where a plugin exposed auth credentials to other users. Making this impossible would be ideal. If we can leverage API keys instead, I believe some of the class Plugin {
private readonly apiKeys = new Map<string, CreateApiKeyResult>();
setup(core, plugins) {
core.http.createRouter().post(
{ path: '/start_job/', },
async (context, req, res) => {
// Note this could be stored in ES if using encrypted saved objects
this.apiKeys.set(jobId, await plugins.security.createApiKey(req));
/* ... */
}
});
}
start(core) {
const esClient = core.elasticsearch.dataClient;
// Very naive polling mechanism for demonstration purposes
setInterval(async () => {
const job = await getJob(esClient);
const apiKeyResult = this.apiKeys.get(job.id);
// TODO: check api key exists and is not expired
// Use API key
const response = await esClient
.asScoped(apiKeyResult.api_key)
.callAsCurrentUser(/* args as normal */);
// TODO: update job
});
}
} |
API Keys are only available when Kibana is communicating with Elasticsearch over TLS. This hasn't been a requirement to use Reporting, so we can't make this switch in a minor version. |
correct. we need to restore this logic #55670
another option could be -
We need to separate them because Spaces & Security plugins require
kibana/x-pack/legacy/plugins/reporting/export_types/common/execute_job/get_custom_logo.ts Line 27 in 4a41c42
Any other examples? Probably we can refactor this code and narrow KibanaRequest | AuthorizationContext to AuthorizationContext ?
Just to note: FakeRequest uses ApiKey to create
|
For what it's worth, I'd much prefer we find an alternative to what is being done here
The two pieces of information from the HTTP request which the security and spaces plugin depend upon is the "base path", as this allows us to determine the current space transparently to end-users, and the Authorization headers, as this is how we later authenticate users against Elasticsearch. I don't want to interject yet another opinion in regard to the specifics of how we want to implement this if it's not helpful. However, if I can be of assistance or you'd prefer I take a different role than commenting on our general needs, please just let me know. |
disclaimer: I haven't caught up on the conversation in this issue yet
++ I also want to find an alternative here. We put this logic in place when migrating Spaces to the NP in order to maintain compatibility with Alerting and Reporting, which at the time were still entirely in the LP IIRC, and therefore relied on the LP convention of retrieving the base path, which is via the Since these background/async interactions don't have a true HTTP request associated with them, they end up crafting this fake request. Prior to the NP conversion efforts, the way to fake the request was to create an object with the |
Totally agree. Are we planning to make these alerting APIs "stable"-ish for 3rd parties yet? If not, then we can use fakerequest for now and break the API once we introduce scoped clients. |
We're planning to have the server side APIs stable / GA in 7.10. By exposing a fake request, the best the alerting team could do is making it clear in the documentation what changes are coming and why it's exposed in the meantime. That way developers are aware how much debt they'll take on until 8.0 or something. |
This came up in conversation today with @lukeelmers. Some of the initiatives the AppArch team is working on will rely on authenticating via API Keys, so without scope-able clients they'll have to take a similar approach to what Alerting has done with FakeRequests, at least for the time being |
Based on the latest discussion in #87990 (comment), it sounds like alerting may soon be providing executors access to the FakeRequest to facilitate the alerting-from-discover effort. In this particular case it is because some of the data plugin's services are now scoped using a |
Un-assigning myself as I'm not actively working on it. |
Elasticsearch relaxed restrictions on API keys usage without HTTPS elastic/elasticsearch#76801 |
Every time when Kibana needs to get access to data saved in elasticsearch it should perform a check whether an end user has access to the data.
Regular mode
This check relays heavily on Security that authenticates a user via elasticsearch API and set
authorization
header accordingly. Having user authenticated Kibana performs all requests to elasticsearch on behalf of the user by attaching authorization headers to an incoming request:Note: the whole Request object is not required to hit elasticsearch API, but something with interface
{ headers: { authorization: 'token, [...other authorization headers: string]: string } }
.FakeRequest mode
There are a couple of cases when the current model doesn't play well
headers.authorization
contains valid credentials.In Reporting plugin
kibana/x-pack/legacy/plugins/reporting/export_types/common/execute_job/decrypt_job_headers.ts
Line 18 in 2f029e6
And in Upgrade Assistant
kibana/x-pack/legacy/plugins/upgrade_assistant/server/lib/reindexing/worker.ts
Line 162 in 4c3dfa3
In all those cases we 'mimic' Request object, although we need only
authorization
headers to be present.Discuss
SavedObjects
. Probably there is something else that I missed.asScoped()
? Are those use cases forFakeRequest
are going to stay in a long-term? Should we operateRequest
term at all? Probably we could rename it to something less confusing, for exampleOwnerCredentials
to emphasise that something with scope-able interface is a valid input.FakeRequest
pattern to perform access data? @mikecoteFakeRequest
to perform a call and declare dependencies on the passed credentials.On the other hand, the ergonomic suffers as we have to use
asScoped
and cannot leverage coming Handler RFC API to extend a request for custom needsThe text was updated successfully, but these errors were encountered: