Collection of horrible python scripts I hacked together to deduplicate files in a foundry world, delete the duplicates and rewrite the foundry databases so the world doesn't break. Optional script to rewrite all the internal .jpeg
, .jpg
and .png
files to .webp
. Unsure how/if it deals with symlinks and/or Unicode characters. Use at your own risk.
WARNING: THE WEBP CONVERTER MIGHT INTRODUCE ISSUES WITH CASE-SENSITIVE FILESYSTEMS.
Tested on/with:
- BG:DiA, CoS, DIP
- Foundry 0.6.6.
- Python 3.9, 3.8.1
- Win 10 x64
- Make a backup!!
- Place all the scripts one folder above the world you want to torment.
- Run
duplicates.py
with the worlds root folder as argument. This will create a textfile calleddedup.txt
with original filename and duplicates listed as absolute paths:
C:\Users\trfunk\Desktop\worlds\bgdia\token\img1.png,C:\Users\trfunk\Desktop\worlds\bgdia\avatar\img3.png
C:\Users\trfunk\Desktop\worlds\bgdia\token\img1.png,C:\Users\trfunk\Desktop\worlds\bgdia\token\img124.png
^ the scripts are in the world folder, bgdia folder is the argument
- Fix any hardcoded shit like the worlds root folder, twice.
- Run
rewrite_and_remove.py
withdedup.txt
as argument. - If you don't want to convert the images to webp, go to the
data/
folder and rename any.db2
files to.db
. Overwrite if needed. Else skip this step.
- Use VanceCole's bulk-convert-to-webp.ps1 powershell script to convert all
.png
files to.webp
(converts.jpegs
,.jpgs
and.gifs
aswell). Make sure this step is finished and worked before you continue with step 7. Else your world will be corrupted. - Fix some hardcoded shit like the world's root folder name, (twice). If you renamed the the
.db2
files to.db
you will have to either rename them back, or change the script. - Run
webp_db_fixer.py
with the world's root folder as argument. The script will only rewrite and delete.jpeg
,.jpg
and.png
files to.webp
, since it doesn't deal with animated gifs yet. - Go to the
data/
folder and rename any.db3
files to.db
. Overwrite if needed. Can remove all.db2
files. - Edit
world.json
in the world's root folder and rewrite the background value from.png
to.webp
.
- Run
remove_empty_folders.py
with the world's root folder as argument. - Before running this again, make sure to remove
dedup.txt
sinceduplicates.py
will just append the duplicate files at the end of the file - resulting in unintended behaviour.
Done.
duplicates.py bgdia/
rewrite_and_remove.py dedup.txt
bulk-convert-to-webp.ps1
webp_db_fixer.py bgdia/
remove_empty_folders.py bgdia/
list_used_media.py > usedmedia.txt
Make sure to rename the .db3
files afterwards and edit world.json
.
- Fix the horrible hardcoded mess.
- Integrate .webp conversation
- .mp3, .flac to opus .ogg?
- Animated .gif to .webm? (And deal with gifs in general)
- Scan DBs for unused media files and list them (maybe delete aswell?)
- Merge into one file
- VanCole for his macros
- Stackoverflow (https://stackoverflow.com/a/38514560, https://stackoverflow.com/a/36113168, https://stackoverflow.com/a/23488980)