UI proposal: as a Dataverse user, I want autocompletion for (long) controlled vocab metadata

[poikilotherm]

We want our researchers to add datasets and annotate them with some metadata needed for research information systems / science reporting:

1. Select one or more departments out of a list of ~70 or maybe even worse, a list of ~500
2. Select one topic out of a list of ~120

Adding the controlled vocabulary via custom metadata TSV files is fine. Choosing from endless checkbox or dropdown is a nightmare...

I would like to propose changing controlled vocabulary metadata fields to autocomplete UI components instead. PrimeFaces offers them out of the box: https://www.primefaces.org/showcase/ui/input/autoComplete.xhtml

This is kind of blocker for us, as we are not very fond of going into production with a bad UI experience for our researchers. 😒 Skipping the metadata in a first production version might be a solution, but it is far from ideal, as people need to add it later (most likely wont happen).

I'm happy to craft a code contribution, but am aware that this needs to be discussed by UI/UX people first. Thinking of @TaniaSchlatter, @djbrooke and @dlmurphy here...

---

This definitly is connected to #4772 (:wave: @RightInTwo, @doigl and @4tikhonov) and might be part of its solution. Mentioning @bronger, too.

[4tikhonov]

Hi @poikilotherm, can you please provide also link to your Search API endpoint to get JSON out from this query?

[poikilotherm]

Sorry, I can't. There is no public available JSON output for this. 😒

[RightInTwo]

Could the ORCID search API be a good example?

https://pub.orcid.org/v2.1/search/?q="müller"?&start=0&rows=10 (with header "Accept: application/json")

#4236 #4772

[poikilotherm]

Guys maybe I need to emphasize a bit: this issue is not about using an API in the background to fill in the data, but using existing views, using autocomplete (multiselect) components instead of generating checkboxes and dropdowns.

@pdurbin requested adding some screenshots to be added here to have a showcase. I'll see what I can do about this.

[RightInTwo]

@poikilotherm Yes, but a general way to include data from APIs via autocomplete would also solve your issue, wouldn't it? That way, you would only need a small API endpoint and not sync your lists to TSV files. Not?

[poikilotherm]

Definitely. But solving both at the same time seems to be a big change which will require a lot more effort. I know that (most?) people at IQSS like small changes, thus I hope (as outlined above) this can be a first small step into the right direction. We need to get things done now... 😉 Nonetheless we do have use cases for #4772 so this should be solved, too. But right now this smaller change blocks us from going into production, while the API autofill is a future feature IOHO.

[pdurbin]

Yes, small chunks, please!

@poikilotherm it looks like there are 180 languages, so that might be a good one to demo your proposed UI solution on. Here's a screenshot:

@jggautier might be aware of a controlled vocabulary with even more values.

[jggautier]

I think this is implied in this issue's title, but might be helpful to make explicit: Would a PR for this issue include a way for installations to specify which type of component (auto complete, drop down, check boxes, etc) is used for which fields? I would say we would want to use autocomplete in addition to other types of components.

[poikilotherm]

@pdurbin thank you, excellent find! Thx for saving me from working on a complete example... 😄

@jggautier I think this should be discussed.

• Dropdowns could be done with autocomplete fields as you can see in the Primeface showcase.
• Checkboxes might be replaced by autocomplete fields with multiselect.

It all depends on what you UX/UI guys prefer. As far as I can foresee, enabling additional rendering options might require a change in the TSV file format or other criteria.

Currently rendering for text field with allowControlledVocabulary=TRUE is done as checkbox when allowmultiples=TRUE and as dropdown when allowmultiples=FALSE. Adding a third rendering option would need better configuration or some other (configurable) criteria like number of terms.

[jggautier]

Ah, thanks, I'd just assumed that the Primefaces "Autocomplete" was the kind of field where someone enters letters and the field simply displays options from a controlled vocab. I hadn't looked at the components @poikilotherm mentioned earlier, which include multi select and dropdown.

I think the languages field @pdurbin pointed out is a great example for these types of components because it's reasonable to assume that depositors know the names of languages they're looking for, so they'll know what to start typing.

For a field like Subjects with its list of 13 terms, I'm not so sure people would know what to start typing. Perhaps the dropdown could include a multi-select?

I'm excited to see Dataverse have autocomplete 😄; like you wrote it's been a blocker for a while for improving how people add metadata.

[poikilotherm]

I just took a look at two other standard Primefaces UI components:

• SelectCheckboxMenu
• SelectOneMenu

<p:selectOneMenu> is already in use at metadataFragment.xhtml and filtering can be activated for easier lookup of values with a search bar.

Instead of <p:selectManyCheckbox> the component <p:selectCheckboxMenu> could be used, offering client side filtering.

Both do not offer limiting the number of elements to choose from (so long lists may still be a nightmare to see on the screen). Could this be an interesting minimal alternative compared to using <p:autoComplete>?

Obviously, preparing a solution geared towards #4772 would better be using autocomplete, which could hook up to search other services later on.

[TaniaSchlatter]

In general, oneMenu.xhtml is good for 3-12 items, checkboxMenu.xhtml is good for 10 to maybe 40 or 50 items, and autoComplete.xhtml is good for 20-many (100?) items. For more than 150 or so items I'd use a categorized search, like Amazon.

I would want to understand the parameters of what might be displayed to help make a choice. We don't want to add a lot of conditional code for what widget to display when.

[djbrooke]

Closing in favor of #6030 to evaluate as part of a larger effort (though I see potential that we could reopen this as a smaller chunk).

[poikilotherm]

I already started some implementation work for this to have sth. you guys could take a look at and play with... I'm feeling blocked. 😐 Meh.

[pdurbin]

I already started some implementation work for this to have sth. you guys could take a look at and play with... I'm feeling blocked. 😐 Meh.

@poikilotherm and I started talking about him feeling blocked at http://irclog.iq.harvard.edu/dataverse/2019-07-18#i_101305 and he has many JSF questions that I wasn't able to answer at the exact times he asked (heads up to @scolapasta and other that the questions are in the IRC log above).

In short @poikilotherm was actively working on a prototype of what's possible using JSF and PrimeFaces and I encouraged him to continue working on his branch so that I can spin it up and demo it to @TaniaSchlatter @mheppler @djbrooke @scolapasta and @jggautier (the design meeting regulars).

The reason @poikilotherm is working on this, of course, is that he feels it's very important before his institution goes live with Dataverse. The "poikilotherm / FZJ" board can be found at https://github.com/orgs/IQSS/projects/4

[djbrooke]

Sure, as I mentioned in the above message, we can re-evaluate this as a smaller effort.

[4tikhonov]

I just spoke to @mbamouni and his institution INRA requires the same functionality.

[alejandratenorio]

@poikilotherm, could you tell me the status of this prototype? We require the same functionality and it's likely that we can do a code contribution

[pdurbin]

Over in pull request #6356 we are playing around with swapping out the component for "Subject" to look like this (not merged yet, could probably use some design work):

You can also see it under "Language", which looks like this:

For comparison, here's how "Subject" and "Language" look on the demo site (which runs code that has already been released):

[mheppler]

ATTN: @poikilotherm, @RightInTwo, @doigl, @4tikhonov, @bronger, @mbamouni, @alejandratenorio, et al.

We had a bug related to the Subject metadata form component reported in issue #6339, so we decided to do away with the scrolling list, and use the selectCheckboxMenu PrimeFaces component that Oliver suggested above.

This new component provides a filter at the top of the dropdown menu, allowing users to easily find controlled vocabulary values in long lists. (See screenshot.)

I am going to close this issue, and ask that you follow the progress of that new issue, Subjects disappear when clicked in metadata editing #6339. We hope to have this improvement included in our next 4.18.1 release.