Improve the usability of keywords for tagging

[claell]

Is your suggestion for improvement related to a problem? Please describe.
Currently, there seem to be two options for tagging: groups and keywords. Currently, groups seem to be the most usable option. However, I wonder, whether keywords are actually the better way to tag documents. However, their current usability is very bad (for example, showing all documents with a keyword requires some search skills: keywords=test).

Describe the solution you'd like
Have keywords act like tags that can easily be used, possibly extending groups.

Additional context

[ThiloteE]

I assume, you want to have a better UI for keywords? Could you provide some examples how it should look like?
Currently JabRef provides keyword fields in some entry editor tab.

[Siedlerchr]

Otherwise you can also create automatic groups based on keywords, if that helps you

[claell]

Yes, I am talking about a better UI for keywords, I know about both, the option to edit them and the option to create groups based on keywords.

What I was thinking of is something like labels here on GitHub (how you add them, how they display on issues and how you can click them or search for them). If needed, I can add more detailed explanation with screenshots.

Possibly, there are even better ways (for the context of reference management) to provide a keyword UI, but that is what I can think of right now.

[claell]

When searching for keywords, I was more thinking about GitLab, than GitHub, actually:

[Beingmani]

Hello,

Can someone help me to replicate this screen in jabref, I have a solution that I can propose to improve the usability of the keywords. I have the design ready. I just need to check how it will look in jabref.

I tried replicating, Couldn't able to find the keyword screen which @claell shared

[ThiloteE]

@Beingmani, Claell did not share this screen from within JabRef, but this is how GitLab implemented keywords. It was just an example of how it could be.

[ThiloteE]

This is how JabRef currently implements keywords:

You then can use the search bar and search and search for specific keywords (e.g. "test") by writing keywords=test.

[ThiloteE]

For an implementation that follows #8739 (comment), #7423 would be very much related.

@mchellelm, how are you faring so far with your implementation?

[ThiloteE]

What I would like to see is keywords showing directly in the maintable (or in a separate specific area in the UI that is easily accessible) and then I want to be able to click on it, which will lead to a place that shows me all my entries with this specific keyword. Just like I can do it here on GitHub with JabRef issues.

Example for how GitHub is doing it:

[mlep]

To manage keywords, JabRef has also a dedicated window, accessible through the menu Edit -> Manage keywords.

About a better keyword management:
While BibTeX does not define the field keywords, biblatex does (as a special field). See page 30 of https://distrib-coffee.ipsl.jussieu.fr/pub/mirrors/ctan/macros/latex/contrib/biblatex/doc/biblatex.pdf). It specifies the comma as a separating character (Note: JabRef allows for other separating characters to be defined in the preferences). In biblatex, the field keywords has a special use. For example (taken from the doc), one can do \printbibliography[section=2,type=book,keyword=abc, notkeyword=xyz].
If keyword management is revised, we should make sure it remains compatible with the biblatex use.

The field groups is JabRef-specific. So no constrain about its use by bib(la)tex. Note that biblatex has the concept of groups (page 333 of doc).

[ThiloteE]

Thank you mlep. This was an important note. I think there should be a separation between keywords and labels. Labels should make it easier for the user to manage the entries. The BibLaTeX field Keywords does too, but there is an important distinction: Keywords may potentially be parsed with TeX engines and printed in the bibliography (depending on the citation style). Labels would not, as bib(la)tex has not defined that field, right?

Therefore, I propose to introduce the "Labels" field to JabRef and it to be JabRef exclusive (comparable to the groups field)

[mlep]

Being a special field in biblatex, the content of keywords is "usually not printed" (according to the doc). A user could decide otherwise by altering a citation style (like with any other field), but I guess we should not take it into account.

I believe having labels, groups and keywords will be mostly redundant. Before defining the field labels, we should define the specific use of each these fields.
We could display keywords (or groups) as a label (like in github). In fact there is already something like this in JabRef: if a group as a color, the column "groups" of the entry editor shows a bar with this color.

[Beingmani]

Labels are mainly user-specific, which are more or less similar to Groups in general. It's like Grouping entries under a similar label. isn't?

And keywords are more specific to entries, Searching for a particular keywords popups the entries related to it. I have a rough sketch of how we can use keywords, let me share it here.

[claell]

@mlep Thanks for the insights. I had a look at page 333 of the biblatex docs, and it looks as if they are talking about different groups (groups of commands?) than what we do here. So not sure whether that is really related.

I agree that there is overlapping between keywords (possibly labels) and groups. Citavi also has both, keywords and groups AFAIK (although their groups are less versatile than the ones in JabRef).

In general, groups tend to "contain" entries, whereas keywords or labels tend to be assigned to entries.

[mlep]

Labels are mainly user-specific, which are more or less similar to Groups in general. It's like Grouping entries under a similar label. isn't?

To me: yes it is, meaning we do not need to introduce the field labels since we have the field groups.

And keywords are more specific to entries, Searching for a particular keywords popups the entries related to it. I have a rough sketch of how we can use keywords, let me share it here.

Another feature related to keywords: currently they can be displayed in the entry editor (as a column).

[mlep]

@mlep Thanks for the insights. I had a look at page 333 of the biblatex docs, and it looks as if they are talking about different groups (groups of commands?) than what we do here. So not sure whether that is really related.

Yes, the name is the same, but the concept is different. So, I do not think there could a confusion between groups in JabRef and groups in biblatex.

I agree that there is overlapping between keywords (possibly labels) and groups. Citavi also has both, keywords and groups AFAIK (although their groups are less versatile than the ones in JabRef).

In general, groups tend to "contain" entries, whereas keywords or labels tend to be assigned to entries.

I agree with your view: the fields groups and keywords have separated usage. What I do not see at the moment: what usage of a field labels that is not already ensured by the fields groups and keywords?

[claell]

@mlep I think, @ThiloteE suggested labels as an additional, JabRef exclusive field to not mess with the keywords field (to not cause unwanted side effects). However, if I read your comments above correctly, there most likely won't be side effects, which is why one can stay with keywords for this purpose.

[claell]

One possibly unwanted side effect that I can think of would be if keywords are also used in DOI or other places (I know that Citavi automatically sets keywords (from a source I currently don't know) when adding an entry). In that case, "official" keywords might conflict with "user" keywords. But not sure whether this really is (or can be) the case.

Edit: Just tested and yes, DOI also gives keywords for some entries. So when retrieving information from DOI, there would be a conflict for this field, if a user puts own values in there.

[ThiloteE]

Being a special field in biblatex, the content of keywords is "usually not printed" (according to the doc).

Very good.

The only other sideeffect I can think of now:

Adding keywords and then exporting this bibliographic data to other people will lead to them having my "personal" keywords I use to manage stuff in their library. By the way, this is also a problem for groups and attached files, but for those, it is easier, because I can just copy the entries to a new library and then to remove all group and file fields from the library. I can easily clean my library. For keywords this approach would not be possible. If I wanted to only export the keywords that come with importing from ISBN / DOI or some other identifier, I would need to go through all the entries manually.

[Siedlerchr]

And to fully mess up the distinction, you can also create automatic groups by keywords

[ThiloteE]

To summarize:

Possible changes:

• 1. Show groups, keywords (and potentially labels) somewhere in the maintable. (something similar to Improve the usability of keywords for tagging #8739 (comment))

  • 1.1 Allow users to click on them.
    • 1.1.1 Upon clicking a group, the user will automatically open the group
    • 1.1.2 Upon clicking a keyword, all entries with that keyword will show in either main-table or in a separate window (e.g. in the edit > Manage keywords window). Also, when within a group, only show entries from that group.
      • 1.1.2.1 Solution A: This approach would use the search feature of JabRef.
      • 1.1.2.2 Solution B: Automatically create a group by keyword when clicking on the keyword.
    • 1.1.3 Upon clicking a label all entries with that label will show in either the main-table or in a separate window. Also, when within a group, only show entries from that group.
      • 1.1.3.1 Solution A: use search feature
      • 1.1.3.2 Solution B: create create group by label

• 2. Improve search for groups, keywords, (potentially labels) to something similar to what is shown in Improve the usability of keywords for tagging #8739 (comment). To do this, maybe Improve search interface #7423 has to be implemented first.

[claell]

Agreed.

Based on that, I think that the UX of the possible solutions should be discussed (for example possible conflicts for keyword usage, etc.) to select a good path forward.

[mlep]

Should this appear in the maintable?
Or should this be included in a redesigned maintable (#6857)?

[ThiloteE]

@mlep I like the current maintable view a lot. I am not so much a fan of the proposed redesign (because of #6857 (comment) and because one probably would have a harder time sorting and finding entries. The excel like view really helps a lot for sorting and ordering), so naturally I personally would prefer this to be added to the current maintable. Others might be of different opinion and that is fine. How about both? :)

@claell To be honest, I think this issue here does not need a lot more discussion, I think we mentioned the most advantages and disadvantages. We just need somebody who implements it. Further discussion can be done in the actual pull request. The longer the discussion goes, the more bothersome for someone who actually wants to work on this because they have to read all this stuff.

[mlep]

@mlep I like the current maintable view a lot. I am not so much a fan of the proposed redesign (because of #6857 (comment) and because one probably would have a harder time sorting and finding entries. The excel like view really helps a lot for sorting and ordering), so naturally I personally would prefer this to be added to the current maintable. Others might be of different opinion and that is fine. How about both? :)

I am quite happy with the current maintable too. My concern is about "1. Show groups, keywords (and potentially labels) somewhere in the maintable.": will there be enough room? For example, displaying the keywords in the maintable (with the current design) do not allow to see them when except the column is pretty wide (in my databases, the list of official keywords is often longer than the title of the paper). Could the label be displayed on multiple lines for each entry?
However, the proposed redesign solve this issue.

[claell]

@ThiloteE regarding discussion: I think there are open points needing decisions, but yes, they can be discussed during implementation phase.

Personally, I don't like the current table view that much, so thanks for the reference of the redesigned maintable issue @mlep :)
Some ideas seem to make most sense with that, but maybe they can also be transferred to the current table design.

[sjb-ch1mp]

G'day! I can see that there's been some discussion in this thread, but that the issue is currently unassigned. If possible, I would like to have a stab at this because it sounds like an interesting problem. I have had a preliminary look at the code base and I have some confidence that I might be concoct a solution. I will be happy to submit a draft pull request when I've made some progress so that you can review the direction that I'm headed in, if you'd prefer. I will be sure to take #7423 into account when I'm devising a way forward.

[ThiloteE]

Sure! You are welcome! :-)

As a general advise for JabRef newcomers: Please check out https://github.com/JabRef/jabref/blob/main/CONTRIBUTING.md for a start. Also, https://devdocs.jabref.org/getting-into-the-code/guidelines-for-setting-up-a-local-workspace is worth having a look at. Feel free to ask if you have any questions here on GitHub or also at JabRef's Gitter chat.

[sjb-ch1mp]

Thanks @ThiloteE. I hope you don't mind the slight delay - I will be getting started on it this weekend!

[sjb-ch1mp]

Apologies @ThiloteE, I think I bit off more than I can chew with this one. I'm going to unassign myself so I don't waste anymore time. I will look for a more 'newcomer-friendly' issue, because this is far more complex a problem than I first anticipated.

[ThiloteE]

Fair enough. I will declare this as "free to take" again.

For anybody else, who might want to have a shot at this: It's not like the issue has to be complete from the start. We have split the task up into multiple smaller issues (and if necessary could be split up into even smaller ones by the people that know more about the code) and it is good enough, if only one or a few of these smaller issues are worked on. Step by step :-)

[calixtus]

Lowered priority of this issue, as this has some larger preconditions, that have to be met first.

• New concept for list main table #6857
• Lucene search backend #8963
• Implement ChipView for the search jabref-koppor#341 or Improve search interface #7423

[ryan-carpenter]

While BibTeX does not define the field keywords, biblatex does (as a special field). ... It specifies the comma as a separating character (Note: JabRef allows for other separating characters to be defined in the preferences).

Note: Biblatex also allows alternative separators, which is helpful when the keywords contain meaningful commas, as in chemical numbers and MEDLINE subject headings.

Separated value fields A separated list of literal values. Examples are the keywords and options fields. The separator can be configured to be any Perl regular expression via the xsvsep option which defaults to the usual BibTeX comma surrounded by optional whitespace.

(from https://mirrors.ibiblio.org/CTAN/macros/latex/contrib/biblatex/doc/biblatex.pdf)