Vanilla3 to Flarum - Clean up User Avatar Folder Structure

[MrCaspan]

If you are like me Vanilla was not our first forum software. We migrated from Vbullien to Vanilla and now to Flarum. I had an issue that there was files all over the place and in sub directories for avatars. For some reason Vanilla wants to make a sub folder for each user to upload their avatar in. The folder and the file have random names. It's very messy and no real reason I see for it. So after Migrating to Flarum I decided to make sure all my photos were in the root of the /assets/avatars/ folder as this is where Flarum stores its avatars. (Flarum uses [a-z][A-Z][0-9]{16}.ext for it's avatars. So time to do some clean up!

First copy over your userpics from Vanilla to Flarum into the /assets/avatars/ folder

Then run the next command to move all the files to the root

// This will move all files in any folder below the one you execute this command in
// run this in your /assets/avatars/ folder 
find . -mindepth 2 -type f -exec mv -t . {} +

Now all the avatars are in the /assets/avatars folder. Next I want to remove all the thumbnail version of all avatars. In Vanilla [n]=thumbnail [p]=profile photo (Larger) The Porter tool updated the Flarum users table to reference the p version so the n verison is not needed

// Remove any file in this folder that starts with `n`
// Run this in your /assets/avatars/ folder 
find . -type f -name "n*" -exec rm -f {} +

Clean up directories

// Run this in your /assets/avatars/ folder 
find . -mindepth 1 -type d -exec rm -rf {} +

Now all my files are in the root of the /assets/avatars, Now it is time to update the database to remove any path that is before the file name

### Run this on the Flarum database
UPDATE users
SET avatar_url = SUBSTRING_INDEX(avatar_url, '/', -1)
WHERE avatar_url IS NOT NULL
  AND avatar_url NOT LIKE 'http%';

Avatars are cleaned up now and in one directory!

[linc]

Nice tip.

Note the reason for the subfolders is an optimization for forums with millions of users, because storing that many files in a flat directory makes the read times too long. Randomly dropping them into 1K folders speeds it up considerably. Of course, if you only have a few thousand users, it's annoying I suppose if you're looking at it, but most folks never will.

[MrCaspan]

Ahh gotcha.. So if I have a million users you are saying its better to have a million folders with 2 files in each IE the P & N files then it is to have 2 million files in the root? I guess that is a bunch of files in a single DIR, to be honest I always think that file systems are better today I guess for Vanilla it was because of the 2 files required for both it would double the number of files. I wish they would have just created a /thumbnail folder to store them in and the profile one in the root!

[linc]

When I left, it only created folders 000-999, so you'd never have millions (it capped at 1K). It looks like that changed in 2020 for shallow reasons, so now you can have as many folders as users, which is bananas. 🤷 Doubt the engineer considered the ramifications.

[MrCaspan]

Okay I thought it was maybe me not understanding so I am glad that it seems crazy. Thanks for looking into that, good information to know!

[linc]

All he had to do was pick the first 2 characters instead of using all 12 and it would have accommodated both issues. 🫠 Oh well, it's a dead project now anyway.