Enhancement: "Pool" Downloads

[GoogleCodeExporter]

This Booru grabber quickly became the one I went to for a lot of reasons: 
Namely, that it provided very easy ways to batch-download, that it was 
graphical instead of commandline (I'm comfortable with those, but they're still 
pains in the ass), and it's actually being updated, unlike most of the other 
ones. (It's even crashing less, which is always a bonus!)

There is one thing, however, that one of those scripts did that this one 
currently does not, as far as I know: Pools.

I'm fairly sure I don't need to explain it, but just in case, I will go over it 
in brief: A pool is a group of images centered around a certain theme, without 
being as specific as keywords. It's a part of both danbooru and gelbooru-based 
boorus, so it should be supported on pretty much all of the sites. An example 
would be this pool, called "Nice Knowing You": 
http://safebooru.org/index.php?page=pool&s=show&id=16

If this feature is a part of the program, then I apologize in advance. If not, 
hopefully it can be added without too much trouble, as then I'd have no reason 
to use anything other than this for my booru-snatching needs. :)

Original issue reported on code.google.com by evolutionrevolution@gmail.com on 20 Jul 2012 at 11:35

[GoogleCodeExporter]

First, thanks for using the program and the compliments. :)
I didn't used other script much, and I don't use pools very much, so I didn't 
know they had that functionality.

But I think you can currently do it on danbooru-based boards, using the meta 
"pool:{id}". For example, for the danbooru pool "Disgustingly Adorable" 
(http://danbooru.donmai.us/pool/show/903), with ID 903, using as tag "pool:903" 
and then batch downloading, it would download the pool. But it doesn't work on 
gelbooru-boards, which doesn't support the "pool:{id}" meta. :/

I guess I could add support of "pool:{id}" on gelbooru boards pretty easily for 
the next version though, so at that time you won't need to use other tools. :D

Original comment by bio.nus@hotmail.fr on 20 Jul 2012 at 11:51

• Changed state: Accepted
• Added labels: Type-Enhancement, OpSys-All
• Removed labels: Type-Defect

[GoogleCodeExporter]

Great. :) I didn't think it'd be too much trouble to add realistically, but I 
know full well that oftentimes, the stuff that looks easy becomes annoyingly 
hard. :x

I'm not sure how you'd do it, though, in terms of implementing it into the 
program, since obviously, different ID numbers will resolve to different pools 
on different Boorus, so any sort of "regular mass search" like with normal tags 
is right out. Perhaps a separate tab for "Pool Search" where the user selects a 
Booru, enters the ID number, and then it grabs the posts as normal? At least, 
that's what I quickly thought of.

Also, hopefully it will work across the three major booru types 
(danbooru/gelbooru/shimmie) and not just gelbooru ones. Pretty sure you would 
make it "universal" in that sense anyway as a matter of due course, but I'm the 
type who likes to try to get things right the first time. :p

Original comment by evolutionrevolution@gmail.com on 21 Jul 2012 at 2:38

[GoogleCodeExporter]

Well, only danbooru and gelbooru have pools, right ? So since it already works 
with danbooru, I guess I only have to do the technical part for gelbooru (which 
API doesn't handle easy search for/on pools). Or maybe I'm wrong?

About the implementation, I think that since there's no mean to search for a 
pool on two boorus at the same time (they rarely exist on both, and even if 
they do you usually want the one from a specific booru), I don't see any reason 
not to use the current search page: you select the booru from the sources tab, 
and then either you put "pool:your_pool_id" on the search field or you use the 
"Plus" window which would have a new field "pool". Plus that would allow to 
filter easily the pool results with another tags or metas.
What do you think?

Original comment by bio.nus@hotmail.fr on 21 Jul 2012 at 11:12

[GoogleCodeExporter]

Well, the issue with that is that oftentimes, when I'm searching for a specific 
tag, I will pull from multiple boorus, yet as you said, the Pools are generally 
booru-specific. To do it your way, I would have to go into Sources, untick all 
the boorus I don't want (so I can search the booru that has the tag), then do 
my searching, download as desired, and then if I later wanted to search 
multiple tags, I'd have to go back in, re-tick them... it'd be tedious, since 
there's no way to set presets of what boorus you generally like when searching. 
:)

That's primarily why I suggested a simple "Pool Search" tab or whatever - you 
pick the booru from a dropdown list, you enter your pool number, and perhaps 
your ratings/tags filtering or whatever, and there you go. The only problem 
with that solution is that a fair bit of the functionality is bound to be 
duplicated, if the rating/tags stuff is added too.

Your idea does take care of that though, but I would say that if you use the 
Plus window, it could work, but only if there is some sort of option to tick 
like "Search Pool IDs" or something, and when that option is ticked, it 
temporarily overrides whatever the user has Ticked in the Sources button, 
because then this way, users don't have to constantly re-set their preferred 
boorus.

The only problem with that solution is that the user kind of needs to be 
reminded if they're in a Pool search mode, or otherwise they're going to be 
wondering why searches are drastically limited compared to before. :x

Original comment by evolutionrevolution@gmail.com on 21 Jul 2012 at 10:10

[GoogleCodeExporter]

Aaaah, that's right, I didn't though about that. That would indeed be tedious 
to change the selected boorus each time.

I see two options:
- As you say suggested, just adding a "pool search" tab, which would be a 
copy/paste of the current search tab + a droplist to choose only one booru 
(maybe instead of the sources button, or near the search field, I'll see what's 
the best when doing it), and a field to enter the pool ID (in addition to the 
tags field).
- In the current search tab, an option in the "+" part to choose only one 
booru, choice that would have priority over the sources window and would be 
deactivated by default, plus the use of the "pool:{id}" meta.

The second choice seems more complicated when you don't know the "pool:{id}" 
syntax and you have to go to the "+" submenu to set the booru you want, while 
the second choice requires to create a new "pool search" tab to specifically 
search for a pool.

I guess I'll pick the first choice, your suggestion, and just add a second type 
of tab that would be accessible using the "File" menu. :D
I'll start working on this tomorrow, so it'll be included in the next version, 
so if you have any other suggestions for this tab or something, don't hesitate 
ok? I'll be glad to hear (read?) them. :)


BTW, your idea of "boorus presets" also sounds good, I'll see if it is doable.

Original comment by bio.nus@hotmail.fr on 21 Jul 2012 at 11:09

[GoogleCodeExporter]

Nifty! I didn't think you'd add it so immediately, I figured it might be a 
couple of subversions down the line at least. :p

I guess the functionality is pretty simple: You're able to select your booru 
from the drop-down list (including boorus any users have added, of course), you 
enter your pool ID number (or possibly select from a searched listing: see my 
suggestion below), and if you want to filter it further, can have it do some 
refinement based on tags and picture explictness rating. Past that, it 
shouldn't really need too much more, I'd think.

Perhaps the only other "enhancement" I can think of in that sense would be if 
there's a way to pull the list of pools without having to go onto the sites 
themselves? (Either some internal database or something, if it can't be 
automatically done through site queries?) This way a user doesn't need to know 
what their pool ID is for any given Booru, because while it's easy for someone 
who's versed in them to find an ID, it may take a casual user some explanation 
as to how the feature works. They can just set it to, say, Gelbooru, click 
"Search for lists," and after some time it'll pull up the lists. This wouldn't 
need much info - the name of the list, the description if applicable, number of 
images. (It seems that Gelbooru and Danbooru-based boorus have a "privacy" flag 
for Public or Private pools, but I think that only restricts who can add to 
them - access still seems to be public, so for downloading purposes that can 
probably be ignored.) Shimmie seems to have no pools capability (though I did 
learn that apparently Ponibooru will be going dark in the middle of August 
since that's where I checked... so I suppose you could remove it then) and 
neither does Zerochan... though I'd hope that if there's any other booru that 
does use pools, they're able to be added relatively easy.

Not quite sure if that's possible, though - I don't know if the boorus give 
info out in that sense in a way that can work with your program. Admittedly, a 
manual list has its own problems of pools being potentially removed... 
hopefully it can be automated. :)

Original comment by evolutionrevolution@gmail.com on 22 Jul 2012 at 2:52

[GoogleCodeExporter]

Unfortunately, the boorus does not provide any consistent API to list pools, on 
the contrary of images. The only way to get them would be to do more than 300 
requests to the server to get the full list, and would represent a lot of data. 
Same thing for searches. Another option would be to download the list only one 
time, then searching from them, but that wouldn't be practical at all. Even 
auto-update would be tedious. I guess I'll start with only the ID + booru 
choice (only for danbooru and gelbooru boards, indeed), and maybe later add an 
option to search for pools. :)

Original comment by bio.nus@hotmail.fr on 22 Jul 2012 at 9:46

[GoogleCodeExporter]

Fair enough. Obviously we want to make sure we don't use excessive bandwidth on 
the boorus either, of course.

In that case, I suppose that all that can be done for now is simply to instruct 
users how to go to the sites and find the pool IDs. Otherwise it's a pretty 
cut-and-dry thing: Search the pools, filter if necessary, and have the ability 
to download images from or batch download the pools.

Auto-updating and the like is just going to have to wait until a solution can 
be worked out... or if their APIs add the ability to get a pool index or 
something. :p

Original comment by evolutionrevolution@gmail.com on 22 Jul 2012 at 11:38

[GoogleCodeExporter]

Added pool tabs in 3.2.0 on Sep 29, 2012.
Searching for pools may come in a later version, but the lack of any API really 
makes this difficult.

Original comment by bio.nus@hotmail.fr on 2 Dec 2012 at 1:17

• Changed state: Fixed