Autoimport fails to find Firefox profiles & issue with hardcoded profiles path when IsRelative=0

[oleole39]

Hello,

Profiles.ini content

[Profile3]
Name=ABCD
IsRelative=0
Path=/path/to/removable/drive/ABCD

[Install4F96D1932A9F858E]
Default=/path/to/custom/path/my-main-profile
Locked=1

[Profile1]
Name=Main
IsRelative=0
Path=/path/to/custom/path/my-main-profile

[Profile0]
Name=default-release
IsRelative=1
Path=zsq8tck1.default-release

[InstallD087BC9767A4CB84]
Default=1koqf71l.default-nightly
Locked=1

[General]
StartWithLastProfile=1
Version=2

1. buku --ai shows the following (i.e. bookmarks from an empty default profile) despite ~/.mozilla/firefox/profiles.ini contains multiple profiles (cf. above):

   me@comp:~$ buku --ai
   Generate auto-tag (YYYYMonDD)? (y/n): y
   Add parent folder names as tags? (y/n): y
   Import bookmarks from chromium? (y/n): n
   Import bookmarks from Firefox? (y/n): y
   1. Firefox Nightly blog
      > https://blog.nightly.mozilla.org/
      # 2024oct30,firefox nightly resources,menu
   
   2. Mozilla Bug Tracker
      > https://bugzilla.mozilla.org/
      # 2024oct30,firefox nightly resources,menu
   
   3. Mozilla Developer Network
      > https://developer.mozilla.org/
      # 2024oct30,firefox nightly resources,menu
   
   4. Nightly Tester Tools
      > https://addons.mozilla.org/firefox/addon/nightly-tester-tools/
      # 2024oct30,firefox nightly resources,menu
   
   5. Planet Mozilla
      > https://planet.mozilla.org/
      # 2024oct30,firefox nightly resources,menu
   
   6. Get Involved
      > https://www.mozilla.org/contribute/?utm_medium=firefox-desktop&utm_source=bookmarks-toolbar&utm_campaign=new-users-nightly&utm_content=-global
      # 2024oct30,toolbar
   
   
   Auto-generated tag: 2024Oct30
   

2. As for trying it with FIREFOX_PROFILE env variable:

   me@comp:~$ FIREFOX_PROFILE="my-main-profile" buku --ai
   Generate auto-tag (YYYYMonDD)? (y/n): y
   Add parent folder names as tags? (y/n): y
   Import bookmarks from chromium? (y/n): n
   
   Auto-generated tag: 2024Oct30
   

   I understand it can't work since that profile is at a custom path (/path/to/custom/path/my-main-profile) whereas the default path is harcoded in buku's code:

   buku/buku

   Line 2879 in 48bf29e

   ff_bm_db_paths = {s: '~/.mozilla/firefox/{}/places.sqlite'.format(s)

   
   Maybe we could have FIREFOX_PROFILE_PATH instead?

[jarun]

Sorry, we can't keep adding support for custom paths for each browser.
You have the option toe export the bookmarks from the browser manually and then import into buku.

[LeXofLeviafan]

@jarun I think handling absolute paths in get_firefox_profile_names() output would not be much of an issue though? I believe this qualifies as a parsing bug.

When used with the sample file provided by OP, that function produces the following output:

['/path/to/custom/path/my-main-profile', '1koqf71l.default-nightly']

[LeXofLeviafan]

@oleole39

~/.mozilla/firefox/profile.ini

Isn't the correct filename profiles.ini though (with an s)?

FIREFOX_PROFILE="Main"

FIREFOX_PROFILE is supposed to contain a directory name 😅

[jarun]

I think handling absolute paths in get_firefox_profile_names() output would not be much of an issue though?

@LeXofLeviafan We support more than firefox. We would need such provisions for every browser we support.

[LeXofLeviafan]

@jarun other browsers use static paths; only Firefox relies on an additional config file to determine profile location (at least out of those supported).

[jarun]

OK. In that case please raise a PR.

[oleole39]

You have the option toe export the bookmarks from the browser manually and then import into buku.

I hadn't thought of that and could live with it.
Although it does appear to me like a parsing bug, as coined by @LeXofLeviafan.

Isn't the correct filename profiles.ini though (with an s)?

Yes, I've just edited my first post.

FIREFOX_PROFILE is supposed to contain a directory name 😅

Indeed, I've just edited my first post with my-main-profile to fit the example i gave.

When used with the sample file provided by OP, that function produces the following output:

['/path/to/custom/path/my-main-profile', '1koqf71l.default-nightly']

So should we have something like FIREFOX_PROFILE_PATH="/path/to/custom/path/" FIREFOX_PROFILE="1koqf71l.default-nightly" buku -ai ?
Or somehow if you have to find your FF profile's directory name, I guess it doesn't additional effort to know the path of it... Hence FIREFOX_PROFILE could be expecting the full profile path instead of just directory name (although I understand that it is slightly longer to type and would be a change of habit for some, but if you use such bookmarks import feature often you are likely to use an alias anyway).

We support more than firefox.

Note that it would extend buku autoimport feature support to most non-Mozilla Firefox-based browsers (Waterfox, Floorp, Librewolf, etc.) which use the same profile system, but whose default paths obviously do not start with ~/.mozilla/firefox/ .

[LeXofLeviafan]

So should we have something like FIREFOX_PROFILE_PATH="/path/to/custom/path/" FIREFOX_PROFILE="1koqf71l.default-nightly" buku -ai?

No, just pass full path to the directory in FIREFOX_PROFILE – it's handled by the same code which handles output of get_firefox_profile_names() (effectively replacing the call that parses profiles.ini), so an absolute path will be handled the same way (after I add support for those).

[oleole39]

Great, if it helps, I have come with the following helper (yet untested!) to manage FIREFOX_PROFILE in each of those cases:

• Custom profile full path (FIREFOX_PROFILE='/path/to/custom/path/my-main-profile')
• List of all profiles in custom directory (FIREFOX_PROFILE='/path/to/custom/path')
• Custom profile in default directory (FIREFOX_PROFILE='my-main-profile')
• List of all profiles in default directory (when the env variable is not set).

def get_firefox_bm_db_paths(firefox_profile, default_ff_folder):
    """List Firefox bookmarks database paths

    Returns
    -------
    ff_bm_db_paths : {str}
        Firefox bookmarks database paths for all profiles (either default or related to existing FIREFOX_PROFILE environment variable).
    """
    import os

    ff_bm_db_paths = {}

    if firefox_profile and os.path.isabs(firefox_profile[0]) and os.path.isfile( os.path.join(firefox_profile[0], 'profiles.ini') ): # Custom profiles directory 
        ff_folder = firefox_profile[0].strip('/')
        profiles = get_firefox_profile_names(ff_folder)
    elif firefox_profile and os.path.isabs(firefox_profile[0]): # Custom profile path
        ff_folder, profiles = os.path.split( firefox_profile[0].strip('/') )
        profiles = [profiles]
    else: # Default profiles directory
        ff_folder = default_ff_folder
        profiles = firefox_profile or get_firefox_profile_names(default_ff_folder)

    if profiles:
        ff_bm_db_paths = {s: ff_folder + '/{}/places.sqlite'.format(s)
                          for s in profiles}

    return ff_bm_db_paths

The idea would be to use it by replacing current portion of the code:

buku/buku

Lines 2876 to 2880 in 48bf29e

	default_ff_folder = os.path.expanduser('~/.mozilla/firefox')
	profiles = firefox_profile or get_firefox_profile_names(default_ff_folder)
	if profiles:
	ff_bm_db_paths = {s: '~/.mozilla/firefox/{}/places.sqlite'.format(s)
	for s in profiles}

with:

default_ff_folder = os.path.expanduser('~/.mozilla/firefox')
ff_bm_db_paths = get_firefox_bm_db_paths(firefox_profile, default_ff_folder)

...3 times due to default_ff_folder needing to be adapted for each supported platforms.

[LeXofLeviafan]

…That's a bit too complicated of a logic. If you need to specify a custom path anyway, you might as well just specify the path to the profile you want to be imported.

Though your idea is pretty similar to what I did. (…I actually implemented the fix on that same day, but I ended up postponing the commit till now due to lack of time to work on tests n'stuff 😅)

[oleole39]

…That's a bit too complicated of a logic. If you need to specify a custom path anyway, you might as well just specify the path to the profile you want to be imported.

Yep, thanks for addressing the issue!