PxIMSE

Local anime style image search based on DeepDanbooru, CLIP and PaddleOCR

Supported file name formats

Images from Pixiv, PixivBatchDownloader, PixEZ ... , Twitter Media Downloader, yande.re or other sources, but without ID information:

• pixiv: {user_id}_{id}_p{index} (Recommended)
• pixiv: {id}_p{index}
• twitter: twitter_{user_name}(@{user_id})_{date}_{id}_photo
• yande.re: yande.re {id} {tags}

Pre-requisites

• Python 3.10+
• Node.js 20+
• Nginx
• Pre-trained DeepDanbooru model

Databases

• MongoDB
• Milvus
• meilisearch (optional)

Refer to docker-compose.yml

Dependencies

pip3 install -r requirements.txt

You might need to handle torch, tensorflow, and paddlepaddle manually

pnpm install

configuration

Configure python/config.yml and config.mjs

Usage

1. Import image information

Import image information, DeepDanbooru and OCR results to MongoDB

python import_images.py mongo

Import CLIP results to Milvus

python import_images.py milvus

Get information from Pixiv website (optional)

python import_images.py pixiv

Attempt to assign the username from an existing document in the database to a document with the same user ID that doesn't have a username (Indexes must be created on the username and userid fields) (optional)

python import_images.py update

Import text to meilisearch (optional)

python import_images.py meili

2. Start Service

cd python
uvicorn api:app --host 0.0.0.0 --port 8000

pnpm dev

Format of Query Criteria

Exists:
tags.xxx
tags1:xxx (AND)
|tags1:xxx (OR)

Does not exist:
!tags.xxx
!tags1:xxx

Text length:
textlen;>xx

Text content (meilisearch):
m;xxx

tags1 (meilisearch):
mt;xxx

Greater than, less than, equal to, not equal to:
filesize;>1000000
tags.xxx;<=0.9
date;>2000-01-01&<2010-01-01
likeCount;=100
viewCount;!=200

tags are obtained from DeepDanbooru
tags1 are retrieved from Pixiv website (for yandere, from the file name)

Format of Sort

Ascending order： filesize;1
Descending order: filesize or filesize;-1

Credits

Inspired by:

• clip-image-search