Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data folder should be in yunohost.app #15

Closed
anmol26s opened this issue Jun 19, 2018 · 18 comments · Fixed by #138
Closed

Data folder should be in yunohost.app #15

anmol26s opened this issue Jun 19, 2018 · 18 comments · Fixed by #138
Labels
help wanted Extra attention is needed

Comments

@anmol26s
Copy link

The data folder should be in /home/yunohost.app/funkwhale/data because sometimes the media can take lot of space fill up the limited space in /var. /home folder is generally reserved with lot of space to keep all the media. Further its good idea to keep media away from the app core files.
I have faced this issue in Peertube too.
See here YunoHost-Apps/peertube_ynh#8

@Jibec
Copy link
Contributor

Jibec commented Jun 19, 2018

I think this issue is also related to the "music" folder of #16
One hint can be the yunohost.multimedia package: https://github.com/YunoHost-Apps/yunohost.multimedia

@rushsteve1
Copy link

YunoHost also has per-user and shared media storage which should be integrated.
You can see it easily if you install Nextcloud and check off the access home option.

@Jibec
Copy link
Contributor

Jibec commented Oct 20, 2018

@EliotBerriot can we have one folder per user so we can enable this? https://docs.funkwhale.audio/configuration.html#music-directory-path

@agateblue
Copy link

Do you mean per-funkwhale user? No, it's not possible because uploaded files won't be binded to only one user anymore in the next releases, when we have deduplication in place (the same file may be uploaded by X user, but will only be stored once)

@anmol26s
Copy link
Author

But we have a media folder in yunohost that have songs folder which is shared to all the users. User can put the songs in this folder from nextcloud and it is shared to with all other users.
But anyways we should have songs directory in /home/yuohost.app/funkwhale and it should not be backedup while upgrade.

@Jibec
Copy link
Contributor

Jibec commented Oct 25, 2018 via email

@rushsteve1
Copy link

It's more standard for YunoHost applications. And being able to easily manage/download the files themselves is quite useful.

Personally being able to use my existing Nextcloud setup to sync my music into the media folder, both to and from Funkwhale, would be very useful.

@Jibec
Copy link
Contributor

Jibec commented Oct 25, 2018 via email

@Jibec Jibec closed this as completed Oct 27, 2018
@IsidoreIsou
Copy link

IsidoreIsou commented Apr 16, 2019

Hello,
On my YNH server, my / folder is almost full, since all my music imported in FW is in it.
I'm not sure what is the best way to resolve it, so I can add more music and free this space? (this is making a lot of problem on the server)

  • Enlarging the space on the /folder?
  • Moving the FW music folder?

I would love to keep things as simple as possible.
Now I have only a 2TB HDD on a VPS to have to most straightforward config possible.

Thanks a lot

@maniackcrudelis
Copy link

As said by @anmol26s, YunoHost provides a media folder, /home/yunohost.multimedia design to store all media files for all apps.
As funkwhale deals with a library of audio files, those files should be stored in this directory, so it would be available for nextcloud, ampache or any other apps that would deal with audio files.
Using a dedicated directory for this app means users would have to duplicate their files to have them available from another app.

Helpers are available to use this directory painlessly.
https://github.com/YunoHost/yunohost/blob/add_ynh_multimedia_helpers/data/helpers.d/multimedia

@IsidoreIsou
Copy link

For a temporary fix, moving the folder in /home and creating a sym link works perfectly.

@Jibec Jibec added the help wanted Extra attention is needed label Jun 27, 2019
@ImaCrea
Copy link

ImaCrea commented Sep 2, 2019

cc @EliotBerriot I think we need help please 🙏
We'd love to start using it at MailTape collective to share our music discoveries between members but I'm afraid to install it atm. Could this be fixed please please :)

@agateblue
Copy link

Hi @ImaCrea, I'm not sure how I can help here.

The directory for uploaded files can already be customized on Funkwhale's side (via the MEDIA_ROOT and MUSIC_DIRECTORY_PATH environment variables), I think it's YunoHost and/or funkwhale_ynh responsibility to set those paths to the best possible options depending of how a YunoHost installation is partitionned / configured, or to make this customizable maybe? (I'm not sure if this is supported in YNH though).

As I'm not a YunoHost user, developper or packager, I cannot work on this myself. Of course, I'm here to help if the person implementing this need advice regarding those variables and their use, reviewing, feedback, or anything else where my input could be useful :)

@ImaCrea
Copy link

ImaCrea commented Sep 8, 2019

Thank you for your dear help @EliotBerriot :) @Jibec does Eliot's answer help you?

@Jibec
Copy link
Contributor

Jibec commented Sep 8, 2019 via email

@ImaCrea
Copy link

ImaCrea commented Sep 8, 2019 via email

@Sporiff
Copy link
Collaborator

Sporiff commented May 20, 2020

My thoughts on this are as follows, based on this thread (I welcome feedback. I'm very new to Yunohost):

New Installations

New installations should have their directory set to /home/yunohost.multimedia/share/funkwhale/[media/import] (or something similar) by default. We can achieve this by using the .env file to set Funkwhale to use the location by default, replacing the relevant sections in the file at install and creating the directory/setting access using the multimedia helper.

Existing Installations

Existing installs will be trickier given that people may have set their servers up differently, but I would guess that the people who have manually changed their setup will be comfortable with making changes to match this repository. If we're working with the assumption that people generally leave these files in /var/www/funkwhale/[media/import] we can do one of the following:

  • Check if the /home/yunohost.multimedia folder exists and create necessary subfolders
  • Migrate the media and import folders to this location
  • Perform the upgrade as normal with the .env file changed to match the new location

The problem with this approach is that Funkwhale uses the .env to determine where it is looking for files when it performs its database migrations, so you would need to move them before this is done which will lead to doubling up as far as I can tell (leaving files in the backup to be restored to /var/www/funkwhale and also in /home/yunohost.multimedia. The other approach is:

  • Check if the /home/yunohost.multimedia/share/funkwhale folder exists and create necessary subfolders
  • If the folder exists already, set the .env variables to this location. If not, proceed as normal
  • If the folder does not currently exist, perform the migration as normal in the standard location and then use the multimedia helper to perform the necessary symlinking, leaving all files in /home/yunohost.multimedia but having Funkwhale continue to interact with /var/www/funkwhale
  • When the next upgrade is run, the symlinks will be removed and everything will work as if it were a new installation with the proper .env variables

This second approach is more long-winded, but it is a bit more robust as far as I can tell for ensuring that we don't bork existing installations since it will work for both locations.

As I say, I welcome feedback. I'm still learning the ropes here, so it would be good to get an idea of how others would go about this. I'm also curious about what expectations are with regards to deinstallation. Should files remain where they are?

@jmbmkn
Copy link

jmbmkn commented Sep 1, 2020

My thoughts on this are as follows, based on this thread (I welcome feedback. I'm very new to Yunohost):

New Installations

New installations should have their directory set to /home/yunohost.multimedia/share/funkwhale/[media/import] (or something similar) by default. We can achieve this by using the .env file to set Funkwhale to use the location by default, replacing the relevant sections in the file at install and creating the directory/setting access using the multimedia helper.

Existing Installations

Existing installs will be trickier given that people may have set their servers up differently, but I would guess that the people who have manually changed their setup will be comfortable with making changes to match this repository. If we're working with the assumption that people generally leave these files in /var/www/funkwhale/[media/import] we can do one of the following:

* Check if the `/home/yunohost.multimedia` folder exists and create necessary subfolders

* Migrate the `media` and `import` folders to this location

* Perform the upgrade as normal with the `.env` file changed to match the new location

The problem with this approach is that Funkwhale uses the .env to determine where it is looking for files when it performs its database migrations, so you would need to move them before this is done which will lead to doubling up as far as I can tell (leaving files in the backup to be restored to /var/www/funkwhale and also in /home/yunohost.multimedia. The other approach is:

* Check if the `/home/yunohost.multimedia/share/funkwhale` folder exists and create necessary subfolders

* If the folder exists already, set the `.env` variables to this location. If not, proceed as normal

* If the folder does not currently exist, perform the migration as normal in the standard location and then use the `multimedia` helper to perform the necessary symlinking, leaving all files in `/home/yunohost.multimedia` but having Funkwhale continue to interact with `/var/www/funkwhale`

* When the next upgrade is run, the symlinks will be removed and everything will work as if it were a new installation with the proper `.env` variables

This second approach is more long-winded, but it is a bit more robust as far as I can tell for ensuring that we don't bork existing installations since it will work for both locations.

As I say, I welcome feedback. I'm still learning the ropes here, so it would be good to get an idea of how others would go about this. I'm also curious about what expectations are with regards to deinstallation. Should files remain where they are?

Can I reproduce this on a fresh installation of Funkwhale? Do I just change the env var 'MEDIA_ROOT=' to '`/home/yunohost.multimedia/share/funkwhale/' and it will just work, or do I need to do an in place import?

yalh76 added a commit that referenced this issue Mar 31, 2021
@yalh76 yalh76 mentioned this issue Mar 31, 2021
5 tasks
yalh76 added a commit that referenced this issue Apr 16, 2021
* Testing (#130)

* Uprade to 1.1 and fixes Twisted by retrograding it (#129)

* Apply example_ynh

Fix #15

* Fix services

* Fix several merging

* Stick to official install process

* fix path_transversal

* Using option '--log_type systemd' with 'yunohost service add' is not relevant anymore

* Try to fix package_check

* Fix package_check test

* Update check_process

* Trying to fix package_check

* Update env.prod

* remove sleep

* Better service name

* missing datadir

* line 111: upgrade_type: unbound variable

* Implement --is_big

* Fix stopping service from previous version

* Fix upgrade

* Fix upgrade

* Apply example_ynh

* Apply example_ynh

* Fix rights

* Update api.src

* Update env.prod

* Update front.src

* Update manifest.json

* Update README.md

* Update README_fr.md

* Fix chmod

* Fix rights

Co-authored-by: Thomas <51749973+Thovi98@users.noreply.github.com>
Co-authored-by: Éric Gaspar <46165813+ericgaspar@users.noreply.github.com>
@yalh76 yalh76 mentioned this issue Apr 16, 2021
5 tasks
yalh76 added a commit that referenced this issue Apr 24, 2021
* Update README.md

* Test bump to 1.0

* Bump to Yunohost 4 (for Debian upgrade)

* Test removal of nonexistent scripts

* Correct Typo

* Update README.md

* Fix linter warnings

* Fix linter

* Fix linter warnings

* Cleaning up

* Update README.md

* Update to 1.0.1 (#119)

* Update to 1.0.1

* Set new permissions

* Add French Readme

* Readme update

* Update README.md

* Update README.md

* Update README_fr.md

* Update README.md

Co-authored-by: Éric Gaspar <46165813+ericgaspar@users.noreply.github.com>

* Update README_fr.md

Co-authored-by: Éric Gaspar <46165813+ericgaspar@users.noreply.github.com>

* Update README.md

Co-authored-by: Éric Gaspar <46165813+ericgaspar@users.noreply.github.com>

* Update README_fr.md

Co-authored-by: Éric Gaspar <46165813+ericgaspar@users.noreply.github.com>

* Finding an available port

* Uprade to 1.1 and fixes Twisted by retrograding it (#129)

* Fix linter warning : no need to use $is_public

* Clarify custom paths needed to import collections of music files in Readme. (#134)

* Update readme to indicate custom import path 

Yunohost installations of Funkwhale use a different path for file imports than those indicated in the Funkwhale documentation.. (/var/www/Funkwhale/import). We should therefore indicate to the admin user what that path is, and steps necessary to use it.   Addresses #132.

* Fix bash highlighting.

* Update README.md

Co-authored-by: Éric Gaspar <46165813+ericgaspar@users.noreply.github.com>

Co-authored-by: Éric Gaspar <46165813+ericgaspar@users.noreply.github.com>

* Apply Example_ynh (#131)

* Testing (#130)

* Uprade to 1.1 and fixes Twisted by retrograding it (#129)

* Apply example_ynh

Fix #15

* Fix services

* Fix several merging

* Stick to official install process

* fix path_transversal

* Using option '--log_type systemd' with 'yunohost service add' is not relevant anymore

* Try to fix package_check

* Fix package_check test

* Update check_process

* Trying to fix package_check

* Update env.prod

* remove sleep

* Better service name

* missing datadir

* line 111: upgrade_type: unbound variable

* Implement --is_big

* Fix stopping service from previous version

* Fix upgrade

* Fix upgrade

* Apply example_ynh

* Apply example_ynh

* Fix rights

* Update api.src

* Update env.prod

* Update front.src

* Update manifest.json

* Update README.md

* Update README_fr.md

* Fix chmod

* Fix rights

Co-authored-by: Thomas <51749973+Thovi98@users.noreply.github.com>
Co-authored-by: Éric Gaspar <46165813+ericgaspar@users.noreply.github.com>

* Update conf/nginx.conf

Co-authored-by: Thomas <51749973+Thovi98@users.noreply.github.com>

* Update conf/nginx.conf

Co-authored-by: Thomas <51749973+Thovi98@users.noreply.github.com>

* fix

* Fix rights for datadir from previous version

* Update upgrade

* Fix recursive chmod

* Fix import link

* Update README_fr.md

Co-authored-by: Thomas <51749973+Thovi98@users.noreply.github.com>

Co-authored-by: Éric Gaspar <46165813+ericgaspar@users.noreply.github.com>
Co-authored-by: Alexandre Aubin <alex.aubin@mailoo.org>
Co-authored-by: Ciarán Ainsworth <cda@rootkey.co.uk>
Co-authored-by: Ciarán Ainsworth <ciaranainsworth@posteo.net>
Co-authored-by: ericgaspar <junk.eg@free.fr>
Co-authored-by: Thomas <51749973+Thovi98@users.noreply.github.com>
Co-authored-by: Amos Blanton <lightnin@users.noreply.github.com>
yalh76 added a commit that referenced this issue May 19, 2021
* Upgrade to 1.1.1 (#138)

* Update README.md

* Test bump to 1.0

* Bump to Yunohost 4 (for Debian upgrade)

* Test removal of nonexistent scripts

* Correct Typo

* Update README.md

* Fix linter warnings

* Fix linter

* Fix linter warnings

* Cleaning up

* Update README.md

* Update to 1.0.1 (#119)

* Update to 1.0.1

* Set new permissions

* Add French Readme

* Readme update

* Update README.md

* Update README.md

* Update README_fr.md

* Update README.md

Co-authored-by: Éric Gaspar <46165813+ericgaspar@users.noreply.github.com>

* Update README_fr.md

Co-authored-by: Éric Gaspar <46165813+ericgaspar@users.noreply.github.com>

* Update README.md

Co-authored-by: Éric Gaspar <46165813+ericgaspar@users.noreply.github.com>

* Update README_fr.md

Co-authored-by: Éric Gaspar <46165813+ericgaspar@users.noreply.github.com>

* Finding an available port

* Uprade to 1.1 and fixes Twisted by retrograding it (#129)

* Fix linter warning : no need to use $is_public

* Clarify custom paths needed to import collections of music files in Readme. (#134)

* Update readme to indicate custom import path 

Yunohost installations of Funkwhale use a different path for file imports than those indicated in the Funkwhale documentation.. (/var/www/Funkwhale/import). We should therefore indicate to the admin user what that path is, and steps necessary to use it.   Addresses #132.

* Fix bash highlighting.

* Update README.md

Co-authored-by: Éric Gaspar <46165813+ericgaspar@users.noreply.github.com>

Co-authored-by: Éric Gaspar <46165813+ericgaspar@users.noreply.github.com>

* Apply Example_ynh (#131)

* Testing (#130)

* Uprade to 1.1 and fixes Twisted by retrograding it (#129)

* Apply example_ynh

Fix #15

* Fix services

* Fix several merging

* Stick to official install process

* fix path_transversal

* Using option '--log_type systemd' with 'yunohost service add' is not relevant anymore

* Try to fix package_check

* Fix package_check test

* Update check_process

* Trying to fix package_check

* Update env.prod

* remove sleep

* Better service name

* missing datadir

* line 111: upgrade_type: unbound variable

* Implement --is_big

* Fix stopping service from previous version

* Fix upgrade

* Fix upgrade

* Apply example_ynh

* Apply example_ynh

* Fix rights

* Update api.src

* Update env.prod

* Update front.src

* Update manifest.json

* Update README.md

* Update README_fr.md

* Fix chmod

* Fix rights

Co-authored-by: Thomas <51749973+Thovi98@users.noreply.github.com>
Co-authored-by: Éric Gaspar <46165813+ericgaspar@users.noreply.github.com>

* Update conf/nginx.conf

Co-authored-by: Thomas <51749973+Thovi98@users.noreply.github.com>

* Update conf/nginx.conf

Co-authored-by: Thomas <51749973+Thovi98@users.noreply.github.com>

* fix

* Fix rights for datadir from previous version

* Update upgrade

* Fix recursive chmod

* Fix import link

* Update README_fr.md

Co-authored-by: Thomas <51749973+Thovi98@users.noreply.github.com>

Co-authored-by: Éric Gaspar <46165813+ericgaspar@users.noreply.github.com>
Co-authored-by: Alexandre Aubin <alex.aubin@mailoo.org>
Co-authored-by: Ciarán Ainsworth <cda@rootkey.co.uk>
Co-authored-by: Ciarán Ainsworth <ciaranainsworth@posteo.net>
Co-authored-by: ericgaspar <junk.eg@free.fr>
Co-authored-by: Thomas <51749973+Thovi98@users.noreply.github.com>
Co-authored-by: Amos Blanton <lightnin@users.noreply.github.com>

* remove others_var

* no more needed

Co-authored-by: Éric Gaspar <46165813+ericgaspar@users.noreply.github.com>
Co-authored-by: Alexandre Aubin <alex.aubin@mailoo.org>
Co-authored-by: Ciarán Ainsworth <cda@rootkey.co.uk>
Co-authored-by: Ciarán Ainsworth <ciaranainsworth@posteo.net>
Co-authored-by: ericgaspar <junk.eg@free.fr>
Co-authored-by: Thomas <51749973+Thovi98@users.noreply.github.com>
Co-authored-by: Amos Blanton <lightnin@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Extra attention is needed
Projects
None yet
Development

Successfully merging a pull request may close this issue.

9 participants