vaex.settings.main.home not working

[Hasham04]

hello,

I am trying to change where vaex stores files downloaded from s3 and I saw in the documentation that we can do : vaex.settings.main.home = '/home/docs/.vaex but

when I do this

def extract_transform_load(args):
    vaex.settings.main.home=r'C:\Users\my_name\me\.vaex'
    df=vaex.open('s3 path',progress=True,fs_options=options)

when I print vaex.settings.main.home it prints my path but the file is downloaded where the script is located which I think is the default behavior. I know we cannot configure this setting from vaex home directory itself but I am unclear on what this means. Any help is appreciated.

[JovanVeljanoski]

I am not 100% sure on this, but try applying the settings right after you import vaex.

[Hasham04]

any advice @JovanVeljanoski .

[JovanVeljanoski]

Ok I looked into the code, and perhaps there is a bug @maartenbreddels

In the meantime you have two options:

• set up an environmental variable "VAEX_PATH_HOME" pointing to the location where you wanna save stuff
• change the current HOME environmental variable to where you want to save stuff.

Or if you run scripts, you can do something like

VAEX_PATH_HOME=/path/to/folder/.vaex python script.py

I hope we can fix this soon, i imagine it is annoying..

[Hasham04]

I set the VAEX_HOME variable in my environmental variables and it still did not help. But I was able to get the files to download in the right location.

I had to use

vaex.settings.fs.path =r'C:\Users\my_name\me\.vaex'
vaex.settings.main.path_lock=r'C:\Users\my_name\me\.vaex'

instead of

vaex.settings.main.home=r'C:\Users\my_name\me\.vaex'

not sure what vaex.settings.main.home does then. it's supposed to point to the place where .vaex folder is generated which keeps the downloaded files but doesnt seem to work. Thanks for your help @JovanVeljanoski

[JovanVeljanoski]

Please read my message carefully.. i did not say VAEX_HOME..

[Hasham04]

I tried that too didn't work. Sorry forgot to mention that @JovanVeljanoski

[JovanVeljanoski]

It works 100%. I tested it myself before writing the suggestion. I am on the latest version of vaex

…

On Fri, May 20, 2022 at 10:12 AM Hasham Rasheed ***@***.***> wrote: I tried that too didn't work. Sorry forgot to mention that @JovanVeljanoski <https://github.com/JovanVeljanoski> — Reply to this email directly, view it on GitHub <#2053 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AENW4Z3TO4SRILRTERUZTADVK5COPANCNFSM5WF3TAJA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- Jovan

[Hasham04]

I went to the control panel ->system->advanced system setting -> Advanced -> Environmental Variables and set the VAEX_PATH_HOME variable and path for both user and system. It did not work. @JovanVeljanoski I am also on the latest version of Vaex

[JovanVeljanoski]

I don't know how this works on windows.. maybe you need to define this in the conda shell or something.

In any case, above you said you've made it work so that's great. We still need to fix the original point tho.

[Hasham04]

I appreciate your help @JovanVeljanoski. You answering questions has been instrumental in me being able to deploy vaex in production at my shop. Many thanks.

[maartenbreddels]

It should be VAEX_HOME, but the other settings VAEX_CACHE_PATH , VAEX_DATA_PATH and VAEX_FS_PATH get initialized at import time relative to VAEX_HOME. So settings vaex.settings.main.home does not change the paths of the others. Hope that makes sense.

[JovanVeljanoski]

@maartenbreddels there is some bug there.. we need to fix.. the vaex.settings is being ignored

[maartenbreddels]

for home? well, i think the problem is not everything is reading it at runtime, but at import time. So changing at runtime will leave a lot of other places still with the old values, like the cache path.

[maartenbreddels]

See #2101