rebuild specimensearch UI

[dustymc]

Replace all dynamic tables with datatables, retire jtables.

[dustymc]

See eg #2741

There was some talk of customizing the specimen results table. The library we use for that hasn't been updated since 2014, which is ANTIQUE for front-end software. It makes little sense to invest in an obsolete product with serious limitations.

https://arctos.database.museum/place.cfm is using datatables. Along with modern and maintained (updated a couple months ago) software, it's also a much more interactive environment than the current specimensearch-->POST--specimenresults; suggest we adopt similar functionality when we rebuild other stuff, which is also a great opportunity to address #3273

[dustymc]

Expansion of #2745 (comment)

Make forms dynamic; never leave the form (URL) you're searching on (but perhaps toggle visibility or something) - eliminate all 'back button doesn't work' and 'the "use last" library hates the "fancy multiselect" library' kinds of problems, eliminate the 'refine widget,' allow better search form customization. Make searching Arctos a much more interactive process, rather than the current "enter something and hope it works on the next form" (mitigated by a bunch of complicated buttons and widgets that themselves don't always work).

This is coming up more and more often, so

1. Changing title
2. This needs discussed. Is there a better way? Can we figure out a framework, or something that will make both this and going further easier, first? Etc. etc. etc.
3. I'm removing the low priority; needs prioritized by AWG, not sure what that should be, but no longer low

I'm not sure what to do about Milestones - I know how to do the technical stuff, could very much use some help on the people-end.

[Jegelewicz]

I'll jump in and add high priority. I simplified search page would be good for both regular Arctos users and the general public.

Let's not leave this out - #3273 (comment)

I know how to do the technical stuff, could very much use some help on the people-end.

Maybe this is a case where technical stuff leads the discussion - if you build it they will come? Maybe just seeing what you could do would inspire the people to comment. I know it sucks, but I think the bulkload tool experience has shown me that until people have something they can use, it is hard for them to imagine what they might like or not about it.

[dustymc]

Thanks, yes we need to round up all related Issues if we're going to get serious about this.

As above, the "place" search form is the simplest possible implementation of what I have in mind. Search for stuff, it creates/changes results without removing anything from the search. Change one of your search parameters, leave the other 742 of them alone, click a button, get new data. Interactive, yay.

"Icing" - the kinds of things become possible in such an approach- could include customizing the search form without getting a bunch of extra junk or having to start over, perhaps hiding the things you don't want to see (I'm not quite sure that's predictable, but we can try), maybe even clicking stuff in the data to change search parameters and get new data.

[Jegelewicz]

So what I get from this is that I could search, get some results, then search with some extra parameters but ONLY on the results of my previous search? I think everyone would like that...

[dustymc]

then search with some extra parameters but ONLY on the results of my previous search?

Not quite, I think. You'd still be searching everything, but adding a parameter would, I believe, result in that functionality. The big UI change is that you'd just scroll up (or click 'show search pane' or something) to add that parameter, not need to worry about going back or 'use last values' getting mangled or needing to remember what you were looking at or wishing you had a way to copy two things without losing sight of your results or.....

[ewommack]

I like it! It would make it really helpful for when I jump between taxon groups in searches, or want to refine my geographic locality.

[dustymc]

We need SSL before proceeding; this should just use the API, that's very limited without a way to implement and test authentication.

https://github.com/ArctosDB/internal/issues/90

[campmlc]

I agree with a task force. What is the timeline on this, and urgency? It seems this came up rather suddenly.

…

On Wed, Nov 30, 2022 at 5:02 PM dustymc ***@***.***> wrote: * [EXTERNAL]* adds visual clarity as to what's in the section See #2745 (comment) <#2745 (comment)> - that would make things very choppy, and depending in what's turned on the nothing can end up taking more screen space than the something. Maybe that'll ultimately prove necessary, but it's not my preference, and would take away from the "customizability" - users would see a BLA section, like it or not (unless I can figure out some 'if any' magic, but I really don't think that's possible without jumping to a higher-level language which would introduce a whole new set of issues). repeat the fields That's not technically plausible. good time to rethink the categories. I'm up for absolutely anything there, whether we use them as some sort of section headers or not. Let me know if you want a CSV to modify! (And FYI the "core" were formerly "required" - the name changed because there are essentially no requirements on this form - at least not yet....) "recordlimit" is usually two words. Aha! Not in my world, but I don't really care and it's easy to change. #2745 (comment) <#2745 (comment)>, a task force still sounds like a good idea. — Reply to this email directly, view it on GitHub <#2745 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADQ7JBGDOTXZ2YUY63WEH5LWK7TKHANCNFSM4NTEJJQA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[dustymc]

came up rather suddenly

The old form not being default is fairly urgent for stability reasons. It can hang around 'internally' for a few months, probably. (But I'm not sure I'll get too excited about reviving it if something chokes either.) I don't have any hard timeline in mind, but this seems to be at least involved, if not necessarily causing, Arctos' toobs getting plugged with some regularity recently, so some urgency is merited.

Redoing the header and GUID pages is about equally urgent for precisely the same reasons, might be nice if that could be included in any considerations.

[campmlc]

Understood, although it is difficult to do this as well as loan forms, which also urgently need everyone's attention. I'm desperately trying to get the several days of concentrated attention to deal with loan templates, and haven't been succeeding.

…

On Wed, Nov 30, 2022 at 5:22 PM dustymc ***@***.***> wrote: * [EXTERNAL]* came up rather suddenly [image: Screenshot 2022-11-30 at 4 10 45 PM] <https://user-images.githubusercontent.com/5720791/204934930-d5eefcff-d6ea-4800-babc-f625a1c0f09a.png> The old form not being default is fairly urgent for stability reasons. It can hang around 'internally' for a few months, probably. (But I'm not sure I'll get too excited about reviving it if something chokes either.) I don't have any hard timeline in mind, but this seems to be at least involved, if not necessarily causing, Arctos' toobs getting plugged with some regularity recently, so some urgency is merited. Redoing the header and GUID pages is about equally urgent for precisely the same reasons, might be nice if that could be included in any considerations. — Reply to this email directly, view it on GitHub <#2745 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADQ7JBABITXBSCN2Q6P4NY3WK7VVXANCNFSM4NTEJJQA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[sharpphyl]

Can we at least make the commands the same for the geography/places search and the catalog record search.

The catalog record search has these options at the very top and they are repeated at the bottom of the selected fields

The geography search has none of these options at the very top.

The commands come after the core fields and instead of submitting a query you find matches.

Can we make the options and the colors and the location of these commands the same.

[Jegelewicz]

One of the things we should be considering is what the default landing page should look like. IMO, nobody coming to Arctos from a Google search should see this.

We do need some public profile searches set up for people to use with an offer to customize if that's what they want to do.

Also WHY do locality/coordinates have be included in the default results? I don't think everyone thinks this is the most important bit of information associated with cataloged items.

[AJLinn]

Also WHY do locality/coordinates have be included in the default results? I don't think everyone thinks this is the most important bit of information associated with cataloged items.

Agreed. That was the first thing I got rid of when I customized my search. First thing added: media (which I immediately dragged next to my GUID column). Maybe have media default and remove the coordinates?

Specific locality is another essential piece of info.

[dustymc]

options and the colors

This form can be ~20 times as long as the geog form so I think dual controls are justified. Colors are #5316 (comment) and I can change values/text as well.

after the core fields

Only if that's how you've customized.

default landing page

https://docs.google.com/spreadsheets/d/1XOUYr7dqKx_koobGg1tQu98hi_S_SHJMqJsjcuQf8YA/edit#gid=1478594408

Don't change OBJ_NAME without talking to me, 'core' is an expected value (initial search), probably everything else can be adjusted.

WHY

Because they are 'core.' Nothing is technically required (but guid is pretty important so I won't remove it...) and I intend to keep the form that way, otherwise please edit this: https://docs.google.com/spreadsheets/d/1OgfoDESI63q7htC8SeONIMWbvF9cqrIc3sS-k804emE/edit#gid=603795984

OBJ_NAME and SQL_ELEMENT - don't change without talking to me

Nothing involving a function can be made 'core'. Making Media default would need a dedicated issue - we'd need to cache and adjust limits (because that can and will melt your browser, and the victims of that will blame Arctos).

Ask if you have any questions, let me know when that's ready to load back to the DB

everyone thinks

Absolutely nothing in common - of this much I'm sure...

public profile searches set

#2745 (comment)

offer to customize

We can phrase or whatever, but I'm not very willing to consider customization without an account. (We actually need to discuss anonymous access as a matter of sustainability, but not here or now.)

I like SOMEHOW hinting at the idea that there's more but it needs an account, how that's done is very open for discussion.

EDIT - I have a request regarding categories. flatTableName.{taxonomic rank} columns (eg flatTableName.genus) are almost certainly more complex, and therefore less likely to be predictable, than any casual user (and maybe most operators) might be expected to understand. Can we move those into some sort of 'curatorial' slot?

[Jegelewicz]

Offer to customize

All I meant there was they can have the customize the search page option, but that definitely should not be the first thing public searchers see. And I would be totally OK with saying that one has to create an account in order to customize search.

[sharpphyl]

I can now see the first 5,000 results on one screen, but it doesn't seem to actually get the total results.

Can it return more than 5,000 results (on multiple pages) and show what the total number of results is?

Also, will my customization be saved like a profile from one session to another so I don't have to repeat the process?

[dustymc]

return more than 5,000 results

I'd need to see more to say anything very useful, but maybe #5080 - the new form limits results based on what you've got turned on. (And #5354 is a request to pay that cost up front for some stuff.)

will my customization be saved

Yup.

[sharpphyl]

I'd need to see more to say anything very useful,

I search for all records in a particular accession.

If I do this in today's search screen, I get 8,941 records.

I can adjust to the number shown to 5,000 so they appear on two pages.

But in the new version, I only get the first 5,000

I can't access the remaining records.

[dustymc]

Is this from your production login account, and if so is it set up (close to) the way you need?

[sharpphyl]

Not sure what account it's from. This is the link: https://arctos-test.tacc.utexas.edu/cat_records.cfm?guid_prefix=DMNS%3AInv&part_disposition=in%20collection

If it's just my login that's limiting the number of records in the results, then ignore the issue. If there's a search field I should check, let me know.

[dustymc]

See #2745 (comment)

Number of rows is determined by costs. Factored in to that are operator status ("us" can deal with timeouts better than a naive user, I hope), the DB cost of compiling the results (cached things are cheap, things that have to run through functions and etc. aren't), and media (essentially no effect on Arctos itself, but it'll eat your browser and you'll blame Arctos).

I can't use test for tuning, so that's sorta 'guess and hope' until it gets to production, then there are very many possibilities and it's production so I have to be careful in approaching things. The more I know about what users are actually doing, the easier I'm going to get through that process.

One simplification might (I'd have to test) be to simply make expensive stuff unavailable to public users, but I think @ArctosDB/search-ui-task-group believes that everyone should have access to the same tools. (I agree, but reality and resources...)

just my login

Your login should enable you to do whatever you need, not the opposite.

search field I should check

Those aren't considered at this time, only results - which are visible via....

... then 'cost' column

Anyway - I raised the limits for "us" for next release, I'll try to get it out ASAP.

[Jegelewicz]

Something weird is going on here

[Jegelewicz]

and in parts too

[AJLinn]

Yeah, I've been noticing that too...

[dustymc]

Blargh.

The background is limited via media query, almost certainly at the request of @ebraker. It looked fine on my device, must have on hers, but I'm coming to the understanding that anything beyond the simplest CSS works as expected on a maximum of two devices, at least if it's written by me.....

I can simplify, remove the thing that's limiting the width of the stripey-bits.

I can try to do more, throw some other random numbers at it or something, but I'm not sure how productive that is. (We're almost there, or I can chase this forever?)

Or ya'll can find that designer we've been talking about forever, they should come equipped with the tools and the specialized knowledge to easily deal with these things that are at the fringes of my skillset. (They won't be able to do much with the antique header nor the legacy GUID page, so if we go this way it probably makes sense to get the functional parts of those out first.)

[Jegelewicz]

Well, it isn't breaking anything - just looks odd. Not a super high priority I guess.

[campmlc]

I see the same thing in Search, customize with my phone

[campmlc]

[dustymc]

same thing ... with my phone

Definitely not, assuming I understand what's being reported, but you might see something similar.

My goal is to get the core pages - the stuff that's taking Arctos down regularly at this point - FUNCTIONAL on mobile. "Pretty" is very much secondary, and even 'functional' is too much for secondary requests, like customizing. It would be FABULOUS to have everything in Arctos work on any device, but we're not there yet, and I can't see how we might get there with current resources.

[sharpphyl]

The difference in color between records that have not been opened (second entry) and those that I have opened (first entry below) is so minor that I keep reopening records I just finished updating.

Please make the color distinction stronger or give us back the ability to check off records we have edited.

[dustymc]

I think we're done with the megaissue.