"Create index pattern" wizard. #13454

chrisronline · 2017-08-10T19:57:51Z

Replaces #13154
Resolves #12867

This PR builds upon #13154 and adds in the data searching. It does not change any part of the UI or UX.

TODO

Rebase and verify functionality
Test attaching an ID to an index pattern
Re-add the time fields error on the second step
Ensure accessibility concerns are handled, like this one

Release Note: Introducing a new wizard to create index patterns that makes index discovery and matching much easier than before.

From the other PR

Changes

Create a directive for each step in the wizard.
Reorganize files into conventional folder structure.
Rename files with more conventional and consistent naming patterns.
Remove all translations.
Display indices, partial matches, exact matches.
Add loading, empty, and success states.
Add option to include system indices.

Changes to the UX

Loading indices

Searching for matching indices

Submitting the new index pattern

cjcenizal · 2017-08-10T22:53:38Z

Looks like we need to fix the functional tests for this stuff, too. Let me know if you need any help with this!

cjcenizal

Looks great man!

alexfrancoeur · 2017-08-11T12:36:23Z

This PR has hopped around a lot 😄 Love the changes described above. I wanted to make sure my comments and feedback were carried over to the new PR. I believe some of it is still relevant. Linking here: #13154 (comment)

Also adding @skearns64 to the thread.

cjcenizal · 2017-08-11T15:15:29Z

@alexfrancoeur Would you mind copying over just the specific comments you think should block this?

alexfrancoeur · 2017-08-11T15:26:29Z

I already synced with Chris on this and quickly categorized as a Google doc. I split my original feedback into three categories - High priority, usability enhancements and future. I would consider high priority closest to "blockers". Personally I find the usability enhancements equally as important to make this a good experience for new and existing users. Some are likely easier to implement than others. Categorized feedback below.

Highest Priority

Defining an index pattern + filtering

I'm struggling with how this can best work. If you look at the below screenshot, we are saying that an index pattern matches in it's current state. While the text matches the indices, the index pattern won't actually match the indices without a wildcard

The fact that we are showing success and allowing a user to move on to the next step with an error is not the focus here. The focus of this "blocker" should be on how we're filtering the table & creating an index pattern simultaneously. Please see next comment

What if we automatically appended a wildcard as soon as they start typing? A user could remove it if they'd like but at least the initial experience would work as expected. Otherwise, we are specifying a single index which I believe is used much less than an index pattern with a wild card (but may be wrong).

Hide optional ID

I'm not sure we want to advertise the optional ID so much. We prefer users use our unique ID. Can we add an advanced settings option here to hide it a bit? I believe this was in the original PR.

Usability Enhancements

Refreshing of table

There seems to be a refresh of the entire table as I type for an index pattern. Is there any way we can dynamically filter the results in the index table without doing a complete refresh?

Highlighting where index patterns match the indices

It'd be great to show where the pattern matches the index. Any chance we can bold where it matches?

How we surface time filters and present as "optional"

I'm not sure "Optional Time filter field name" is the best header here. Especially given the fact that making a selection here is a blocker for me moving forward. Also, the option to "not use a time filter" is pretty hidden. I wouldn't think to click into this drop down to create an index without a time filter. This is an issue with the current index pattern creation view as well.

What if we called the title "Select time field" where the dropdown provides all available options, with a checkbox that would disable this drop down and allow a user to create an index without a time field. @cjcenizal @snide I believe there is some history here for why this option is in the dropdown. Any additional info you can provide or suggestions for a better approach?

Future PR's

There have been a number of designs from @alt74 around the index pattern management page for K7. On the design side of things, we seem to be in agreement that this information architecture is a good approach. There are still some kinks to work out but at a high level, the single step / side by side approach is preferred. What are your thoughts around the work effort to implement for K6? I'll set aside some time early next week to discuss in more detail.

alexfrancoeur · 2017-08-11T15:53:10Z

@chrisronline @cjcenizal

I've taken a look at the most recent PR. I'd say all of my feedback still stands and I want to add some additional color around the top "blocker" I had mentioned in my comment above.

In the above image, I've typed in an index pattern "logs" and at first glance I do not understand why I can't click next. The table below appears to be filtered on text that I've typed. This is clearly due to the fact that I only have logstash indices but I imagine a new user will be in a similar situation. As we all know, the first reaction from a user is not to read. The warning in blue does not stand out to me as an erroneous index pattern, which I'm assuming is intentional. We shouldn't be showing failures as soon as a user lands in Kibana.

I'd like to propose a solution that I think may work. Upon typing an index pattern initially, we could automatically append the wildcard that is constantly shown as a pre-selected text element. If you were to hit backspace, the first character deleted would be the asterisk and at that point you're on your own and can customize accordingly.

The indices in the table below would simply be filtered as they are today and the text in the input field would be your index pattern. If they choose to remove the asterisk, the table below would be filtered on whatever is in that input box. This promotes a good getting started experience, a full understanding of the index pattern they've created and a quick and easy escape hatch for the edge cases / "advanced" index patterns. With this approach, we could also begin to highlight where the index pattern matches an index in the table.

chrisronline · 2017-08-11T18:09:04Z

Pushed up a couple changes:

Added the wildcard to the search query after the user starts typing
Highlighting the search query in the result set:

Let's see how these feel

cjcenizal · 2017-08-11T18:24:27Z

Nice! Great idea @alexfrancoeur. I really like how the user falls into a successful state from the get-go.

@chrisronline I found a minor bug. In the screenshot below you can see I've typed "log" but it looks like only "lo" has been bolded.

Also, let me know if you need help resolving the merge conflicts. Looks like it's just UI Framework stuff.

chrisronline · 2017-08-11T18:27:09Z

Nice find @cjcenizal! I'll fix that up

chrisronline · 2017-08-11T19:09:18Z

Another update.

Added the request to auto hide the index pattern id controls:

cjcenizal · 2017-08-11T19:41:00Z

...management/sections/indices/create_index_pattern_wizard/step_time_field/step_time_field.html

+        <a
+          class="kuiLink"
+          ng-click="stepTimeField.toggleAdvancedOptions()"
+        >


This isn't keyboard accessible (can't be tabbed to) because it doesn't have an href on it. We just need to add kbn-accessible-click as an attribute to solve this problem.

I'd suggest removing the <small> element and placing this link and the "index pattern name" field beneath the time field filter name field, but I'd also like to hear @snide's thoughts.

Sounds good. I'll wait to hear back from @snide on the latter, but I'll fix up the former.

I actually really like the second suggestion too. I went ahead and added it.

Do you think we should toggle the text between Show and Hide?

Nice! Good idea about switching between "Show" and "Hide" language. If we're going that far, how about using the ToggleButton component from the UI framework instead of a link?

chrisronline · 2017-08-14T15:03:16Z

Added @cjcenizal suggestion around where to put the advanced fields on the second page.

Do you think we should toggle the text between Show and Hide?

chrisronline · 2017-08-14T18:07:56Z

Per @cjcenizal's suggestion, we're going to use the toggle button from the ui framework:

alexfrancoeur · 2017-08-14T19:18:31Z

Hey @chrisronline - loving the progress! I wanted feedback from @cjcenizal and yourself on a thought I had.

In the above gif, you'll notice that I entered an index pattern and then the next screen I see is immediately red because I hadn't selected a time filter from the drop down. It don't believe it's a good experience to see red with one of your firs time in the product. What do you think of either of these approaches?

Intelligently choose a time filter as the time series visual builder does? @timestamp or next possible datetime field. This could then be adjusted if need be.
Don't highlight red or create a warning until you hit "create index pattern"

chrisronline · 2017-08-14T19:49:52Z

@alexfrancoeur That's a great point! I just pushed up a fix so the red border will not appear. It shouldn't ever appear because the field is optional

cjcenizal · 2017-08-14T23:46:59Z

@chrisronline One issue I just noticed is that with this change to remove required from the time field select, is that the "Submit" button defaults to being clickable.

If we go down this path, I think we should remove the "I don't want to use a Time Field" option, which would help this UX make a little more sense.

I think it might also be helpful to add more information to help text for this control when there's no time field selected: "The Time Filter will use this field to filter your data by time. Without a time field selected, you won't be able to use the date picker or create time-based visualizations with this index pattern's data." Then when a time field is selected, this text can revert back to the current text, "The Time Filter will use this field to filter your data by time." @alexfrancoeur What do you think?

alexfrancoeur · 2017-08-15T15:54:50Z

@chrisronline

That's a great point! I just pushed up a fix so the red border will not appear. It shouldn't ever appear because the field is optional

So oddly enough, the input itself is not optional. A user needs to define a timeframe or choose not to use a timeframe. The initial experience & labeling of this is honestly a bit confusing in Kibana today. I had mentioned this in my usability feedback (one of the last bullet points). @cjcenizal also touches upon this in his most recent comment.

What do you two think of this proposal?

Make this input box required and remove the I don't want to use the Time Filter option
Only turn border of the drop down red if the "Create index pattern" page is clicked with no input selected
Under advanced options, add a check box that says "Ignore time filter", "I don't want to use a time filter" or something along those lines
If there is only one datetime field available, we can make it easy for a user and automatically select it

A few other comments:

Thoughts on formatting the title, description and input box a bit differently? I feel as if normally, these types of forms will normally show a title, provide sub-text that describes what goes in the dropdown/input box and then the interactive component
Advanced options needs additional descriptive text, example below.

Custom index pattern ID
Kibana will provide a unique identifier for each index pattern. If you do not want to use this unique ID, customize the ID below.
[ input box ]

We have users with thousands of indices. If we haven't already, we should test how this feature performs with a ridiculous amount of indices to filter through
I know we aren't tackling the auto-suggested index templates for index patterns with this issue, do we have another open for tracking this request?

cjcenizal · 2017-08-15T17:37:47Z

@alexfrancoeur I think that introducing another control to the UI will add more complexity to the form. I think the problem we're trying to solve is to communicate to the user that they are required to choose either a time field or no time field. I think the wording "Optional" in the label is confusing, and the messaging around this control doesn't communicate this requirement. I think we can solve the problem and keep things simple by removing the word "Optional" from the label and by updating the help text with a note like "You can choose not to have a time field, but... {insert caveats here}". What do you think?

In terms of the layout of the form, the "Label -> control -> help text" pattern is what we've established in K6 and we're carrying through to K7. I think we should stick with this for consistency.

I think we should also stick with the pattern of making the submit button active only after the required options have been selected. This is how the first step in the wizard behaves, so I think it should behave the same here, too.

alexfrancoeur · 2017-08-15T19:02:06Z

@cjcenizal sounds good to me. Assuming @chrisronline is on board, I'm 👍 on all 3 points.

👍 I agree that adding additional options in the advanced settings adds more complexity. I'm all for modifying to the text to be more descriptive and am comfortable keeping the behavior as is. We have not had many complaints from the community about this directly it's just always been an awkward experience for myself.

👍 on consistency with the layout in K6 and K7

👍 disabling the submit button if nothing is selected from the dropdown menu

chrisronline · 2017-08-16T14:34:26Z

Great thoughts! Made some updates.

Here is how it will look for an index without any date field:

Here is the flow for an index with at least one date field:

chrisronline · 2017-08-16T16:52:29Z

Tagging @tylersmalley to take a look

cjcenizal · 2017-08-16T23:15:55Z

@chrisronline Looks great! For the first screenshot, I'd just suggest changing the copy to "The indices which match this pattern don't contain any time fields. Add documents to these indices to specify a Time Filter field name." This way the user has an idea of what they're missing out on and what they can do about it.

chrisronline · 2017-08-22T13:42:09Z

@tylersmalley Great feedback! Fixed up and I updated from master so we'll see how the build goes.

chrisronline · 2017-08-22T15:06:42Z

jenkins, test this

alexfrancoeur · 2017-08-22T20:08:37Z

@chrisronline I hope I'm not too late here, and I'm sure your sick of my feedback but I noticed two more issues I'd like to comment on.

We worked so hard to remove the red border on the time filter selection, any chance we do the same for the index pattern page? Same scenario, we're showing an error before the user has even done anything.

I think we can remove "Optional" from the header here. We removed the text everywhere else. "Settings" or "Configure settings" could be some of the options. I prefer "Configure settings" or something along those lines as it's a call to action. Similar to "Define index pattern" seen on the previous step.

chrisronline · 2017-08-22T20:17:01Z

@alexfrancoeur Never too late! Thanks for the feedback. I'll clean that up

tylersmalley · 2017-08-23T16:50:11Z

Addresses #10438 and #10440.

alexfrancoeur · 2017-08-23T19:09:40Z

Is #5053 really addressed? Seems like this is more related to #13582

tylersmalley · 2017-08-23T19:11:14Z

Ah, you're right @alexfrancoeur. Removing from the comment so Github doesn't auto-close.

tylersmalley · 2017-08-24T00:32:05Z

@chrisronline, changes look good. Just one thing I previously missed, can we debounce the index list? Even 100ms would help.

tylersmalley

LGTM - debouncing the index list would be nice, but I don't see it being a requirement.

* Create Index Pattern Creation wizard. - Create a directive for each step in the wizard. - Reorganize files into conventional folder structure. - Rename files with more conventional and consistent naming patterns. - Display indices, partial matches, exact matches. - Add loading, empty, and success states. - Add option to include system indices. * Temp: comment out and mock getIndices functionality. * Hook up data seaching * Add for/id connections for form inputs and labels * Automatically append a wildcard * Highlight the index pattern in the results * Ensure we only remove the last character if it's a `*` * Auto hide index pattern id controls * Ensure this link is tabbable * Move the advanced fields down * Use toggle button * This shouldn't ever be required * Revert "This shouldn't ever be required" This reverts commit b6e31e79308271e7f04ea1d42ce66e32e7aa0612. * Update based on comments in PR * Ffew more changes * Port changes from Tyler's PR, #13353 * Remove unnecessary file * Fix broken functional tests * Copy changes * Fix functional tests * Remove loading from the main select, and move to an additional select * Show help text when loading * Fix sorting * Fixing broken functional tests * Couple changes from PR review * Ensure input field does not show a red border until touched * More descriptive and consistent copy

chrisronline · 2017-08-24T13:05:23Z

Backport
6.x: 4885ca1

chrisronline added :Management review v6.1.0 v7.0.0 labels Aug 10, 2017

chrisronline self-assigned this Aug 10, 2017

chrisronline requested a review from cjcenizal August 10, 2017 19:57

cjcenizal mentioned this pull request Aug 10, 2017

"Create index pattern" wizard. #13154

Closed

5 tasks

cjcenizal approved these changes Aug 10, 2017

View reviewed changes

cjcenizal reviewed Aug 11, 2017

View reviewed changes

chrisronline force-pushed the enhancement/index_pattern_page branch from 46508a4 to 9b27cf1 Compare August 16, 2017 18:13

chrisronline added 2 commits August 22, 2017 16:40

Ensure input field does not show a red border until touched

71f2525

More descriptive and consistent copy

db1a73a

tylersmalley approved these changes Aug 24, 2017

View reviewed changes

chrisronline merged commit 9175137 into elastic:master Aug 24, 2017

chrisronline deleted the enhancement/index_pattern_page branch August 24, 2017 13:05

chrisronline mentioned this pull request Aug 24, 2017

[Management] Test and address multi-cluster setups with new index pattern page #13689

Closed

chrisronline mentioned this pull request Sep 12, 2017

Optional index pattern id is not allowing wildcards in new wizard #13956

Closed

This was referenced Oct 27, 2017

[Management] Address UI searching bug with index pattern creation #14646

Merged

Don't fetch mappings at index pattern creation #9692

Closed

chrisronline mentioned this pull request Nov 8, 2017

404 in console while creating index pattern #12979

Closed

jbudz mentioned this pull request Nov 8, 2017

Remove default request looking for logstash-* on pattern creation page #7638

Closed

chrisronline added the release_note:roadmap label Nov 16, 2017

chrisronline mentioned this pull request Feb 13, 2018

[Management] Support alias in index pattern creation #16715

Merged

chrisronline mentioned this pull request Mar 14, 2018

Index Patterns with Commas #17147

Closed

"Create index pattern" wizard. #13454

"Create index pattern" wizard. #13454

Conversation

chrisronline commented Aug 10, 2017 • edited Loading

TODO

From the other PR

Changes

Changes to the UX

Loading indices

Searching for matching indices

Submitting the new index pattern

cjcenizal commented Aug 10, 2017

cjcenizal left a comment • edited Loading

Choose a reason for hiding this comment

alexfrancoeur commented Aug 11, 2017

cjcenizal commented Aug 11, 2017

alexfrancoeur commented Aug 11, 2017 • edited Loading

Highest Priority

Defining an index pattern + filtering

Hide optional ID

Usability Enhancements

Refreshing of table

Highlighting where index patterns match the indices

How we surface time filters and present as "optional"

Future PR's

alexfrancoeur commented Aug 11, 2017

chrisronline commented Aug 11, 2017

cjcenizal commented Aug 11, 2017

chrisronline commented Aug 11, 2017

chrisronline commented Aug 11, 2017

cjcenizal Aug 11, 2017

Choose a reason for hiding this comment

chrisronline Aug 11, 2017

Choose a reason for hiding this comment

chrisronline Aug 14, 2017

Choose a reason for hiding this comment

cjcenizal Aug 14, 2017

Choose a reason for hiding this comment

chrisronline commented Aug 14, 2017

chrisronline commented Aug 14, 2017

alexfrancoeur commented Aug 14, 2017 • edited Loading

chrisronline commented Aug 14, 2017

cjcenizal commented Aug 14, 2017 • edited Loading

alexfrancoeur commented Aug 15, 2017 • edited Loading

cjcenizal commented Aug 15, 2017

alexfrancoeur commented Aug 15, 2017

chrisronline commented Aug 16, 2017

chrisronline commented Aug 16, 2017

cjcenizal commented Aug 16, 2017

chrisronline commented Aug 22, 2017

chrisronline commented Aug 22, 2017

alexfrancoeur commented Aug 22, 2017

chrisronline commented Aug 22, 2017

tylersmalley commented Aug 23, 2017 • edited Loading

alexfrancoeur commented Aug 23, 2017

tylersmalley commented Aug 23, 2017

tylersmalley commented Aug 24, 2017 • edited Loading

tylersmalley left a comment

Choose a reason for hiding this comment

chrisronline commented Aug 24, 2017

chrisronline commented Aug 10, 2017 •

edited

Loading

cjcenizal left a comment •

edited

Loading

alexfrancoeur commented Aug 11, 2017 •

edited

Loading

alexfrancoeur commented Aug 14, 2017 •

edited

Loading

cjcenizal commented Aug 14, 2017 •

edited

Loading

alexfrancoeur commented Aug 15, 2017 •

edited

Loading

tylersmalley commented Aug 23, 2017 •

edited

Loading

tylersmalley commented Aug 24, 2017 •

edited

Loading