Question on permissions

[IzzySoft]

My recently enhanced scanner just reported on yesterday's update of your app:

! repo/org.joinmastodon.android_84.apk declares sensitive permission(s):
  android.permission.REQUEST_INSTALL_PACKAGES android.permission.READ_EXTERNAL_STORAGE*
! repo/org.joinmastodon.android_84.apk contains signature block blobs: 0x504b4453 (DEPENDENCY_INFO_BLOCK; GOOGLE)

READ_EXTERNAL_STORAGE is granted implicitly (indicated by the trailing *) due to WRITE_EXTERNAL_STORAGE. Could you please clarify what this is needed for? Further, and much more problematic is REQUEST_INSTALL_PACKAGES; what apps are you installing there?

As for the DEPENDENCY_INFO_BLOCK, that's easy to get rid of:

android {
    dependenciesInfo {
        // Disables dependency metadata when building APKs.
        includeInApk = false
        // Disables dependency metadata when building Android App Bundles.
        includeInBundle = false
    }
}

For some background: that BLOB is supposed to be just a binary representation of your app's dependency tree. But as it's encrypted with a public key belonging to Google, only Google can read it – and nobody else can even verify what it really contains.

Thanks in advance!

[grishka]

READ_EXTERNAL_STORAGE is granted implicitly (indicated by the trailing *) due to WRITE_EXTERNAL_STORAGE. Could you please clarify what this is needed for?

For saving images from the media viewer. The image is already cached so I just copy it to the external storage from my cache to avoid downloading it a second time. I don't ever use or request READ_EXTERNAL_STORAGE.

Further, and much more problematic is REQUEST_INSTALL_PACKAGES; what apps are you installing there?

App's own updates. Only added for githubRelease/githubDebug flavors.

For some background: that BLOB is supposed to be just a binary representation of your app's dependency tree. But as it's encrypted with a public key belonging to Google, only Google can read it – and nobody else can even verify what it really contains.

What's the point of it being a thing then? 🤔

[IzzySoft]

Thanks @grishka – and apologies for the late reply: it seems GH loses some notifications 😢

• r/w storage: Thanks, set accordingly here

App's own updates. Only added for githubRelease/githubDebug flavors.

Is there an APK available without that self-updater? Like in F-Droid, the inclusion policy in my repo speaks against those.

What's the point of it being a thing then?

You'll have to ask Google, it's signed with their key. Probably something something proprietary-issue-with-Playstore, idk. Apart from Google, nobody can even read it anyway. So being a proprietary blob, it should rather not be inside a FOSS APK 😉 (there's even more to those blobs, but that goes beyond this issue here).

[grishka]

Is there an APK available without that self-updater? Like in F-Droid, the inclusion policy in my repo speaks against those.

Yes, just build the release flavor. The self-updater is the only difference between release and githubRelease.

[IzzySoft]

You forget that i just run a binary repo. No build environment here. So I need to grab the APK from you.

[grishka]

Oof. Well, I plan to setup a CI pipeline eventually, so you'll be able to grab apks from there. Probably even eutomatically. In the meantime, here's the latest one:
mastodon-release.apk.zip

[IzzySoft]

Thanks @grishka – but my updater cannot grab from pipelines. It can grab from releases, from a directory inside the repo – or from "fixed URLs" (which may redirect to the real one). So maybe your CI/release workflow could include that APK with releases?

[grishka]

So maybe your CI/release workflow could include that APK with releases?

I deliberately don't want to make a release apk easily available to users because then, not knowing the difference between flavors, they will install it and get stuck on that version because there's no self-updater.

[IzzySoft]

Understood. Then maybe I remove it from my repo and F-Droid users have to rely on F-Droid.org instead (including the delayed updates)? Alternatively you could make the updater opt-in (if it isn't already): have a bool option for that (initially null), when null ask whether to enable, then store (and allow changing via settings) – including proper explanations? You know, "informed and explicit consent" and all that 😉 Giving you an example from a different app:

[grishka]

Then maybe I remove it from my repo and F-Droid users have to rely on F-Droid.org instead (including the delayed updates)?

That would be cumbersome. If you need a fixed URL, I can set up an endpoint on my personal server that would redirect to the apk from the latest CI run (and maybe move it to joinmastodon.org later, I will need to ask the team). Will that work?

Alternatively you could make the updater opt-in (if it isn't already)

I'm not sure how to properly integrate that into the UI. I definitely don't want a modal that pops up by itself like on your screenshots. Forcefully diverting user's attention like that is one of the worst UX sins for me. Besides, there isn't the usual "they may track you" concern, it's just an unauthenticated GitHub API request. If I were to make a setting for that, it would be opt-out.

[IzzySoft]

If you need a fixed URL, I can set up an endpoint on my personal server that would redirect to the apk from the latest CI run (and maybe move it to joinmastodon.org later, I will need to ask the team). Will that work?

Sure, that would work fine – provided the fixed URL also sends a LAST_MODIFIED header.

Besides, there isn't the usual "they may track you" concern, it's just an unauthenticated GitHub API request

It's not so much the tracking part there, though that also plays into it. Those "side-loaded" updates would bypass all scans performed in my repo – which are not done because I mistrust you, but because everybody can miss something.

If I were to make a setting for that, it would be opt-out.

Which would not be compatible with the inclusion criteria on either F-Droid or my repo. But as said above: a fixed URL providing the last-modified flag would do for me, even long-term if needed. It just needs to be "stable" then: should it disappear, I have no idea where to look – while at releases here I could see at least if the file is gone or just the naming pattern changed 😉 But as I can reach you here in such a case, this would be a good compromise for both of us I'd say?

[grishka]

I got CI working: https://github.com/mastodon/mastodon-android/actions/runs/8117973419

But apparently, it isn't possible to download artifacts as-is. They are put into zip archives even if it's just a single file because reasons. See actions/upload-artifact#14
The API doesn't allow it either:

The :archive_format must be zip.

Are you okay with having to extract the apk from a zip archive?

[IzzySoft]

Are you okay with having to extract the apk from a zip archive?

My updater does not support that unfortunately. But a quick search on "github workflow release apk" returns some interesting results, maybe you could use one of those? example 1, example 2, step-by-step guide

[grishka]

This puts it into releases, which is something I explicitly don't want. I don't want to make non-self-updating apks easily available to users because they will install them and then get stuck on that version. This is why only githubRelease ones go there.

I suppose I can have a cron job webhook handler on my server that would periodically check for new CI artifacts and download and unzip them if there are any.

[IzzySoft]

This puts it into releases

I saw that. But not being familiar with Github actions/CI I thought maybe that step could be replaced somehow 😢

I don't want to make non-self-updating apks easily available to users because they will install them and then get stuck on that version.

That could happen via the F-Droid website (or mine) then as well – though it's less likely there than it might be with the Github releases page, yeah… Thanks for looking into a viable approach, @grishka!

[grishka]

That could happen via the F-Droid website (or mine) then as well

The idea is that because F-Droid manages updates for apps it installed, it's perfectly fine to have a non-self-updating apk there.

[IzzySoft]

Sure, got that. Same with my repo 😉

[grishka]

I haven't made the thing on my server yet, but you can grab the apk for 2.4.1 from here: https://github.com/mastodon/mastodon-android/actions/runs/8176605132

[IzzySoft]

Shall I disable automatic update checks in my repo then until the final link becomes available? Else the APK would be replaced from here should you make a new release. Won't happen of course if I put the new one into the repo before it shows up here.

Oof. Cannot download via wget as that seems to be some JavaScript circus. Have to click the link in the browser, store locally, then go where it's stored, unzip, then push the file over… OK, have replaced it now, thanks!

[IzzySoft]

@grishka ^^ (just in case the notification didn't reach you – looks like several got lost for me and others as well).

[grishka]

Shall I disable automatic update checks in my repo then until the final link becomes available?

Yes, it would make sense

[IzzySoft]

To not fall back too far, I've set it to "Static" now (which means "monthly checks" instead of "daily checks" of my updater). So the next check would be in 14 days from now (static run is on the 1st of each month around 2 am UTC).

Does that give you sufficient time? I just want to ensure my repo doesn't "fall too much behind" should this issue be "forgotten" (buried in the heap of work).

[grishka]

Yes

[grishka]

GitHub's API documentation seems to contradict reality 🤔

[grishka]

So apparently this is a "known issue" that's been ignored for 4 years... yikes.
actions/upload-artifact#51

[IzzySoft]

Like several others 😢 There are still notifications lost. Releases disappear (you can see them when you have the URL, but they are not listed at the release page; you can reach them via their tags, though; this happens when you delete a release and re-create it with the same name, in some cases)…

[IzzySoft]

@grishka just seeing this issue is still open. May I ask what the status is? There was an update in May (so not sure if "Static" shall still apply), but REQUEST_INSTALL_PACKAGES is still there.

[grishka]

but REQUEST_INSTALL_PACKAGES is still there.

Where? It is in the githubRelease flavor because it needs to be able to update itself. I can't remove it from there.

[grishka]

I submitted a bug report to GitHub about that artifact download endpoint requiring authentication.

[IzzySoft]

Thanks! Hope we can get that alternative download working.

[IzzySoft]

Yuck, warning popped up again. Any chance we can get this solved, @grishka? I cannot keep it up forever this way as it violates the inclusion criteria. Afraid next time I'll have to disable/remove the app 😢

[grishka]

Here's the last response I got from GitHub support:

Hi Gregory,

Thanks for confirming.

I agree that this is unexpected behaviour, so I'll escalate this and get the documentation updated accordingly too.

I'll keep you posted with any updates, but if there's anything else you need in the meantime just let me know.

Kind regards,
Dan | GitHub Support Engineer

Maybe I should research request authentication. Maybe they have some sort of non-personal token that only grants read access to public data.

[IzzySoft]

Thanks @grishka – and I'll keep my 🤞 Nasty situation…

[grishka]

Okay, I managed to get it working. I didn't get the token scopes right the first time because the layout of that form is so confusing.

The always-latest apk is at https://grishka.me/uploads/mastodon-release-latest.apk, the update-checking script runs twice a day.

[IzzySoft]

Thanks so much, @grishka – that worked out fantastic! I've adjusted the config to pick the APK from there, making sure Fastlane is still checked from here, manually triggered an update – and all went smooth. REQUEST_INSTALL_PACKAGES is gone, the blob was already gone before, so all is solved now 🤩 Closing the issue then.