integrate offline LLM into gallery

[shamirahim111]

#19

like the issue says, there is an app called tidy that does this OOTB so i was hoping if the community is interested in this feature native to the app.

[gmasil]

A fourth option was mentioned: download the (large) network from the gallery to enable this feature. The feature itself could be included for relatively little more storage needed for those who don't want/need this feature. This also allows to independently update the network and even for the user to load thirdparty/selfmade networks with different categories.

Edit: just saw that it was your idea :)

[naveensingh]

The on-demand feature modules were introduced for this same reason but then Fdroid users won't get this feature.

We can't do in-app downloads because we don't use the internet permission. Of course, almost all apps now use internet permission but I think most apps with internet access will sooner or later start sharing data with third parties.

An extension app seems like the only solution.

[inson1]

I think it would be great to see some graph or something to know how fast could there be lighter solution - maybe it isnt even problem - maybe before there is time to implement it, there could be ligher solution already, plus I dont think its worth it developing whole new extension app, because for example of one year wait

[inson1]

@naveensingh what do you think?

[jherazob]

Hardly anything even needs an LLM, i want my gallery to be just a freaking gallery, not a chatbot, please do not start bloating it with unnecessary garbage

[haldi4803]

It's not LLM that we need but CLIP.
Example
https://github.com/greyovo/PicQuery

You can generate a database of all your pictures takes around 1-20 seconds per picture, depending on your device. And then Search the Database for queries with relative low RAM usage.

It's s pretty cool feature to find all pictures of Food or Cars by simply searching for it.

Immich has perfected the implementation. With proper Search for Tags, Date Location...
immich-app/immich#1269
But that might be a lot of work for a smartphone app.

[naveensingh]

I naturally assumed the author meant "a small model that can generate tags given an image ", not a large language model like the GPT.

A simple implementation would involve using object detection models to generate a list of objects present in an image, there are libraries for that like https://developers.google.com/ml-kit/vision/object-detection. Models like OpenAI's CLIP take it to the next level.

[haldi4803]

I'm not up to date.
Last time I checked those object detection systems work pretty good for a surveillance camera. But pretty bad for image classification/description.
That's where clip had a huge advantage on proper search results.

[Panak252]

In that regard it doesn't seem like a bad idea. However, I believe it is quite involved and may take lots of time to implement such a feature, which isn't (the way I see it) a priority.
If it is integrated, allowing the user to toggle it is something desirable (may for example allow to rebuild the tag database when a new update for it is released).

[Panak252]

I don't think an LLM is something right, something that should go with a gallery app. If its task is to be a gallery with a couple neat features, I would consider text recognition, though that is quite a feat to acomplish. An LLM seems overkill and not in line with a gallery app's goal, the way I see it. :)

[naveensingh]

Closing since there's already an issue about it: #19