Slow start in Firefox

[xuhcc]

Firefox can't load any tab for a 2-3 minutes after browser start when IPFS companion is enabled.
Problem does not occur in Chromium.

OS: Arch Linux
Firefox: 64
IPFS companion: v2.6.3

[lidel]

• Are you using any other extensions in Firefox?
• Are you able to confirm the issue is present when starting with empty profile that has only ipfs-companion installed?
  • You can run empty profile by disabling remote interface and providing custom --profile:

    $ mkdir -p $HOME/empty-directory
    $ firefox --no-remote --profile $HOME/empty-directory 'https://addons.mozilla.org/en-US/firefox/addon/ipfs-companion/'

• Does the Browser Console (ctrl+shift+j) show any errors?

[xuhcc]

• Yes, I have a big number of other extensions. Disabling them did not help.
• The problem does not occur with empty profile.
• There are several errors, but nothing seems relavant (I see the same errors with empty profile):
  

[lidel]

Thank you. What if you install ipfs-companion into empty profile and restart the browser (exit and run again against the same --profile directory)?
Is the issue of slow startup still there?

[xuhcc]

Only old profile is affected, and I think the problem is caused by the file addonStartup.json.lz4 in the profile directory. When I remove this file, FF loads quickly but unfortunately it forgets all preferences from other extensions.

[lidel]

I was about to suggest using Refresh Firefox feature, but it will clear extension data as well.

Are you on MS Windows? Did you try solution from https://support.mozilla.org/en-US/questions/1229621#answer-1159980? Not sure if there is any other way to fix this, or anything we can do on companion side.

If you want to get to the bottom of this try reporting an upstream issue with addonStartup.json.lz4 at https://bugzilla.mozilla.org, perhaps they will know what to check next.

[da2x]

I may actually have the same problem on Fedora Linux 29. New tabs are unresponsive for 30-180 seconds. I didn't even suspect the IPFS Companion extension. Disabling the IPFS Companion and restarting Firefox makes the problem go away.

[xuhcc]

I did more experiments and found that startup delay is proportional to the number of items in storage/default/ subdirectory of Firefox profile. Delay could be reduced by clearing offline website data (FF > Preferences > Privacy & Security > Clear History > Select only 'Offline Website Data' and time range 'Everything'). Other notes:

• I'm on Linux (Arch Linux distro).
• The problem occurs only when I run Firefox for the first time after turning on my laptop. If I close FF and start it again, it loads quickly.
• Profile directory is located on HDD, not SSD. I suspect that delay is caused by large number of disk I/O operations.
• It is not clear how addonStartup.json.lz4 is related to this problem. Sometimes removal of this file had no effect.

[lidel]

@da2x are you able to replicate the issue with an empty profile? (with steps from #655 (comment)). @xuhcc suggested the problem occurs only on older profiles.

Would be extremely useful to find a reproducible way to trigger this issue (I've run out of ideas myself – unable to reproduce on my two boxes).

[da2x]

It does reproduce in a new profile for me after installing the extension. I’ll do some more testing and see if I can come up with better steps.

[lidel]

FYI I just released v2.7.0 (Stable) – make sure you test with the latest version

[xuhcc]

v2.7.0 still causes delay on browser start. But I have found that it doesn't happen if I remove these lines from add-on/src/lib/ipfs-companion.js:

browser.webRequest.onBeforeSendHeaders.addListener(onBeforeSendHeaders, { urls: ['<all_urls>'] }, ['blocking', 'requestHeaders'])
browser.webRequest.onBeforeRequest.addListener(onBeforeRequest, { urls: ['<all_urls>'] }, ['blocking'])
browser.webRequest.onHeadersReceived.addListener(onHeadersReceived, { urls: ['<all_urls>'] }, ['blocking', 'responseHeaders'])

[lidel]

@xuhcc @da2x does the delay happen when local HTTP Gateway (go-ipfs) is running, or only when it is offline?

[da2x]

It only happens when the local gateway is running. On Windows, I can also see unexpectedly high resource usage from the Windows DNS Client. After some digging I found a way to cut the number of required DNS queries for DNSLink discovery in half (pull request ipfs/go-ipfs/#5950). While this will probably make DNS server operators and edge gateway devices happier, it doesn’t appear to address the browser slowdown nor Windows DNS Client’s high resource usage. It should yield a decent performance improvements on high latency connections, though.

I see a way to optimize onBeforeSendHeaders: The following check should happen when registering the filters the listener instead of forcing all_urls through this function:
https://github.com/ipfs-shipyard/ipfs-companion/blob/c90114b159177edb05a6a926478e835dc219776e/add-on/src/lib/ipfs-request.js#L118

[lidel]

Thanks, this is very useful and makes sense: DNSLink is the most expensive thing we do, so it is heavily cached, but when browser starts DNSLink cache is empty and every new hostname triggers a lookup.

Will try to optimize, but would be useful to reproduce this, eg. by throttling DNS responses.
@da2x Question:

• do you have many tabs open? If so, how many triggers issue on browser start?
• which DNSLink policy you had selected in Companion's Preferences?

[xuhcc]

@lidel In my case startup delay did not depend on the state of local gateway. I tried to determine what code in request handlers could cause it, but without success. And then suddenly delay disappeared when Firefox Deleloper Edition (which I use for debugging) updated from version 66.0b2 to 66.0b3, so maybe it was a browser bug after all.

[lidel]

Could be. In recent builds of Firefox Nightly 66 they finally moved storage.local to be backed by indexedDB, instead of serialized to a file in the profile directory:

• Bug 1488825: Enable ExtensionStorageIDB backend on all channels

It is possible there was a regression related to those changes which caused the slowdown.
Let's keep this open until Firefox 66 hits stable channel.

[xuhcc]

Closing, as the bug was in Firefox itself.