Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Search Assistant] Use scopes to modify behavior contextually #195785

Merged
merged 15 commits into from
Oct 11, 2024

Conversation

sphilipse
Copy link
Member

@sphilipse sphilipse commented Oct 10, 2024

Summary

This actually uses the Search Assistant scope to modify the assistant's behavior depending on the context they're in. The assistant now:

  • Defaults to Observability mode
  • Is a Search assistant in the Search pages
  • Switches dynamically, changing available functions, prompts and instructions based on context

@sphilipse sphilipse added release_note:skip Skip the PR/issue when compiling release notes Team:Search v8.16.0 labels Oct 10, 2024
@sphilipse sphilipse requested review from a team as code owners October 10, 2024 14:31
@botelastic botelastic bot added ci:project-deploy-observability Create an Observability project Team:Obs AI Assistant Observability AI Assistant labels Oct 10, 2024
@elasticmachine
Copy link
Contributor

Pinging @elastic/obs-ai-assistant (Team:Obs AI Assistant)

@sphilipse sphilipse added the backport:version Backport to applied version labels label Oct 10, 2024
Copy link
Member

@afharo afharo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just 2 minor comments. Approving to unblock from the Core POV.

x-pack/plugins/search_assistant/server/functions/index.ts Outdated Show resolved Hide resolved
x-pack/plugins/search_assistant/server/functions/index.ts Outdated Show resolved Hide resolved
@elastic-vault-github-plugin-prod elastic-vault-github-plugin-prod bot requested a review from a team as a code owner October 10, 2024 15:16
Copy link
Member

@pheyos pheyos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

x-pack/test_serverless/tsconfig.json changes LGTM

@sphilipse sphilipse requested a review from a team as a code owner October 11, 2024 12:30
@neptunian
Copy link
Contributor

Is there a plan to separate out kibana privileges? Currently if a user doesn't have access to observability AI Assistant, they won't have access to the Search assistant.

@sphilipse
Copy link
Member Author

sphilipse commented Oct 11, 2024

Is there a plan to separate out kibana privileges? Currently if a user doesn't have access to observability AI Assistant, they won't have access to the Search assistant.

Yes, I've got a task to set up a separate feature for the Search assistant and have one grant access to the other (because they're the same thing, really).

I'll do that in a separate PR.

@neptunian
Copy link
Contributor

neptunian commented Oct 11, 2024

Screenshot 2024-10-11 at 9 55 19 AM
  • NIT: I noticed that we already use 'scope' in the Observability AI Assistant settings to define where it appears in Kibana. It might be a bit confusing to use 'scope' in a different way here for distinguishing AI assistants. We could consider changing the wording in the settings to avoid overlap. Alternatively, instead of using 'scope' for differentiating the AI assistants, something like AssistantContext or geAssistantContext might help clarify their specific roles instead of scope.

  • Since Search AI Assistant doesn't have it's own settings, instead of the Observability AI Assistant showing up in Discover for example when I change scope to "everywhere", the Search assistant will if that was the last assistant you had open as the assistant scope isn't changing when navigating outside of either observability or security. I think this is unexpected behaviour for a user setting their observability ai assistant to show up "everywhere". But perhaps it makes sense to use the last assistant you were on. Is the idea to have a generic unified assistant settings to replace the observability specific one?

Looks good otherwise, thanks!

@neptunian
Copy link
Contributor

Couple other things:

  • Should we add a functional test that test scope switching between Search, Observability, and neither (Discover) within Kibana?
  • I wasn't able to add/retrieve anything from the knowledge base using the Search assistant.

@sphilipse
Copy link
Member Author

Thanks for the reviews and thoughtful comments @neptunian!

NIT: I noticed that we already use 'scope' in the Observability AI Assistant settings to define where it appears in Kibana. It might be a bit confusing to use 'scope' in a different way here for distinguishing AI assistants. We could consider changing the wording in the settings to avoid overlap. Alternatively, instead of using 'scope' for differentiating the AI assistants, something like AssistantContext or geAssistantContext might help clarify their specific roles instead of scope.

The scope I'm adding isn't user-facing so I'm not very concerned about the confusion. We can always change the wording if we find ourselves getting confused.

Since Search AI Assistant doesn't have it's own settings, instead of the Observability AI Assistant showing up in Discover for example when I change scope to "everywhere", the Search assistant will if that was the last assistant you had open as the assistant scope isn't changing when navigating outside of either observability or security. I think this is unexpected behaviour for a user setting their observability ai assistant to show up "everywhere". But perhaps it makes sense to use the last assistant you were on. Is the idea to have a generic unified assistant settings to replace the observability specific one?

Yes, the long-term plan is to have one settings page across all assistants. For now we'll adjust the observability settings to be "for observability and search" in a follow-up PR. I also have a follow-up task scheduled to set the scope based on some more complicated rules involving solution nav, where it was opened last, and where the user is now. I'll address that next week. Once that's in I'll also add a functional test to make sure the scope changes correctly.

I wasn't able to add/retrieve anything from the knowledge base using the Search assistant.

Yes, that's not a concern because that separate page will be going away as well--we don't want to surface the search assistant on its own page, only in the flyout.

@sphilipse sphilipse enabled auto-merge (squash) October 11, 2024 15:39
Copy link
Contributor

@kc13greiner kc13greiner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@elasticmachine
Copy link
Contributor

elasticmachine commented Oct 11, 2024

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

  • [job] [logs] Jest Tests #5 / ShowMoreButton calls onShowMoreClick on button click
  • [job] [logs] Jest Tests #15 / Transform: Minimal initialization

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id before after diff
observabilityAIAssistant 106 111 +5
observabilityAIAssistantApp 251 380 +129
searchAssistant 172 303 +131
total +265

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
@kbn/ai-assistant 62 63 +1
@kbn/ai-assistant-common - 3 +3
observabilityAIAssistant 292 296 +4
total +8

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
observabilityAIAssistant 17.7KB 19.2KB +1.6KB
observabilityAIAssistantApp 153.7KB 238.2KB +84.4KB
searchAssistant 96.9KB 179.2KB +82.2KB
total +168.2KB

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id before after diff
observabilityAIAssistant 28 27 -1

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
observabilityAIAssistant 46.2KB 46.9KB +650.0B
observabilityAIAssistantApp 8.7KB 8.7KB +56.0B
searchAssistant 4.9KB 4.9KB +1.0B
total +707.0B
Unknown metric groups

API count

id before after diff
@kbn/ai-assistant 62 63 +1
@kbn/ai-assistant-common - 3 +3
observabilityAIAssistant 294 298 +4
total +8

ESLint disabled line counts

id before after diff
@kbn/ai-assistant-common - 1 +1

Total ESLint disabled count

id before after diff
@kbn/ai-assistant-common - 1 +1

History

@sphilipse sphilipse merged commit ee341d5 into elastic:main Oct 11, 2024
46 checks passed
@kibanamachine
Copy link
Contributor

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/11300452200

@kibanamachine
Copy link
Contributor

💔 All backports failed

Status Branch Result
8.x Backport failed because of merge conflicts

You might need to backport the following PRs to 8.x:
- Extract AI assistant to package (#194552)
- [KB] create @kbn/product-doc-artifact-builder package (#193847)

Manual backport

To create the backport manually run:

node scripts/backport --pr 195785

Questions ?

Please refer to the Backport tool documentation

sphilipse added a commit to sphilipse/kibana that referenced this pull request Oct 12, 2024
…c#195785)

## Summary

This actually uses the Search Assistant scope to modify the assistant's
behavior depending on the context they're in. The assistant now:
- Defaults to Observability mode
- Is a Search assistant in the Search pages
- Switches dynamically, changing available functions, prompts and
instructions based on context

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
(cherry picked from commit ee341d5)
@sphilipse
Copy link
Member Author

💚 All backports created successfully

Status Branch Result
8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

@sphilipse sphilipse added backport:version Backport to applied version labels and removed backport:version Backport to applied version labels labels Oct 12, 2024
@kibanamachine
Copy link
Contributor

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/11306209404

kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Oct 12, 2024
…c#195785)

## Summary

This actually uses the Search Assistant scope to modify the assistant's
behavior depending on the context they're in. The assistant now:
- Defaults to Observability mode
- Is a Search assistant in the Search pages
- Switches dynamically, changing available functions, prompts and
instructions based on context

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
(cherry picked from commit ee341d5)
@kibanamachine
Copy link
Contributor

💚 All backports created successfully

Status Branch Result
8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

kibanamachine added a commit that referenced this pull request Oct 12, 2024
…195785) (#196014)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[Search Assistant] Use scopes to modify behavior contextually
(#195785)](#195785)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Sander
Philipse","email":"94373878+sphilipse@users.noreply.github.com"},"sourceCommit":{"committedDate":"2024-10-11T23:09:06Z","message":"[Search
Assistant] Use scopes to modify behavior contextually (#195785)\n\n##
Summary\r\n\r\nThis actually uses the Search Assistant scope to modify
the assistant's\r\nbehavior depending on the context they're in. The
assistant now:\r\n- Defaults to Observability mode\r\n- Is a Search
assistant in the Search pages\r\n- Switches dynamically, changing
available functions, prompts and\r\ninstructions based on
context\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"ee341d5f801ca42ed26acf0544b0bc59948d0214","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","Team:Search","Team:Obs
AI
Assistant","ci:project-deploy-observability","v8.16.0","backport:version"],"title":"[Search
Assistant] Use scopes to modify behavior
contextually","number":195785,"url":"https://github.com/elastic/kibana/pull/195785","mergeCommit":{"message":"[Search
Assistant] Use scopes to modify behavior contextually (#195785)\n\n##
Summary\r\n\r\nThis actually uses the Search Assistant scope to modify
the assistant's\r\nbehavior depending on the context they're in. The
assistant now:\r\n- Defaults to Observability mode\r\n- Is a Search
assistant in the Search pages\r\n- Switches dynamically, changing
available functions, prompts and\r\ninstructions based on
context\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"ee341d5f801ca42ed26acf0544b0bc59948d0214"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/195785","number":195785,"mergeCommit":{"message":"[Search
Assistant] Use scopes to modify behavior contextually (#195785)\n\n##
Summary\r\n\r\nThis actually uses the Search Assistant scope to modify
the assistant's\r\nbehavior depending on the context they're in. The
assistant now:\r\n- Defaults to Observability mode\r\n- Is a Search
assistant in the Search pages\r\n- Switches dynamically, changing
available functions, prompts and\r\ninstructions based on
context\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"ee341d5f801ca42ed26acf0544b0bc59948d0214"}},{"branch":"8.x","label":"v8.16.0","branchLabelMappingKey":"^v8.16.0$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/196013","number":196013,"state":"OPEN"}]}]
BACKPORT-->

Co-authored-by: Sander Philipse <94373878+sphilipse@users.noreply.github.com>
sphilipse added a commit that referenced this pull request Oct 12, 2024
…195785) (#196013)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[Search Assistant] Use scopes to modify behavior contextually
(#195785)](#195785)

<!--- Backport version: 8.9.8 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Sander
Philipse","email":"94373878+sphilipse@users.noreply.github.com"},"sourceCommit":{"committedDate":"2024-10-11T23:09:06Z","message":"[Search
Assistant] Use scopes to modify behavior contextually (#195785)\n\n##
Summary\r\n\r\nThis actually uses the Search Assistant scope to modify
the assistant's\r\nbehavior depending on the context they're in. The
assistant now:\r\n- Defaults to Observability mode\r\n- Is a Search
assistant in the Search pages\r\n- Switches dynamically, changing
available functions, prompts and\r\ninstructions based on
context\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"ee341d5f801ca42ed26acf0544b0bc59948d0214","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","Team:Search","Team:Obs
AI
Assistant","ci:project-deploy-observability","v8.16.0","backport:version"],"number":195785,"url":"https://github.com/elastic/kibana/pull/195785","mergeCommit":{"message":"[Search
Assistant] Use scopes to modify behavior contextually (#195785)\n\n##
Summary\r\n\r\nThis actually uses the Search Assistant scope to modify
the assistant's\r\nbehavior depending on the context they're in. The
assistant now:\r\n- Defaults to Observability mode\r\n- Is a Search
assistant in the Search pages\r\n- Switches dynamically, changing
available functions, prompts and\r\ninstructions based on
context\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"ee341d5f801ca42ed26acf0544b0bc59948d0214"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","labelRegex":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/195785","number":195785,"mergeCommit":{"message":"[Search
Assistant] Use scopes to modify behavior contextually (#195785)\n\n##
Summary\r\n\r\nThis actually uses the Search Assistant scope to modify
the assistant's\r\nbehavior depending on the context they're in. The
assistant now:\r\n- Defaults to Observability mode\r\n- Is a Search
assistant in the Search pages\r\n- Switches dynamically, changing
available functions, prompts and\r\ninstructions based on
context\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"ee341d5f801ca42ed26acf0544b0bc59948d0214"}},{"branch":"8.x","label":"v8.16.0","labelRegex":"^v8.16.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:version Backport to applied version labels ci:project-deploy-observability Create an Observability project release_note:skip Skip the PR/issue when compiling release notes Team:Obs AI Assistant Observability AI Assistant Team:Search v8.16.0 v9.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants