Implement support for Privacy Manifests

[rolfbjarne]

We should look into what we need to do with regards to Privacy Manifests: https://developer.apple.com/documentation/bundleresources/privacy_manifest_files/describing_use_of_required_reason_api

[takeshik]

Are there any updates of this issue, or is there any action in another repository to address this issue?

Apple writes:

From Spring 2024, apps that don’t describe their use of required reason API in their privacy manifest file won’t be accepted by App Store Connect.

If things go as planned by Apple, .NET apps on iOS might be at risk of being rejected for publication or updates by the store this spring. (though Unity seems to be dealing with it in their own way)

We are seeking some workaround which can be done only by ourselves, without framework support, but it might be impossible.
It would be best if the framework could address this issue, before the store's new regulation applies.

[dalexsoto]

Here is an example of a Framework that contains the file PrivacyInfo.xcprivacy which is just a plist dictionary with the described contents of https://developer.apple.com/documentation/bundleresources/privacy_manifest_files/describing_use_of_required_reason_api?language=objc

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>NSPrivacyAccessedAPITypes</key>
	<array>
		<dict>
			<key>NSPrivacyAccessedAPIType</key>
			<string>NSPrivacyAccessedAPICategorySystemBootTime</string>
			<key>NSPrivacyAccessedAPITypeReasons</key>
			<array>
				<string>35F9.1</string>
			</array>
		</dict>
		<dict>
			<key>NSPrivacyAccessedAPIType</key>
			<string>NSPrivacyAccessedAPICategoryFileTimestamp</string>
			<key>NSPrivacyAccessedAPITypeReasons</key>
			<array>
				<string>C617.1</string>
				<string>3B52.1</string>
				<string>0A2A.1</string>
			</array>
		</dict>
		<dict>
			<key>NSPrivacyAccessedAPIType</key>
			<string>NSPrivacyAccessedAPICategoryDiskSpace</string>
			<key>NSPrivacyAccessedAPITypeReasons</key>
			<array>
				<string>E174.1</string>
			</array>
		</dict>
		<dict>
			<key>NSPrivacyAccessedAPIType</key>
			<string>NSPrivacyAccessedAPICategoryUserDefaults</string>
			<key>NSPrivacyAccessedAPITypeReasons</key>
			<array>
				<string>CA92.1</string>
			</array>
		</dict>
	</array>
	<key>NSPrivacyTracking</key>
	<false/>
</dict>
</plist>

[rolfbjarne]

If we wanted to implement this well, here's an idea:

We add a new attribute:

[AttributeUsage (AttributeTargets.Assembly | AttributeTargets.Method | AttributeTargets.Field | AttributeTargets.Property | AttributeTargets.Constructor | AttributeTargets.Class, AllowMultiple = true)]
public class PrivacyManifestRequirementAttribute : System.Attribute {
	// The value for the NSPrivacyAccessedAPIType key in the privacy manifest
	// For instance "NSPrivacyAccessedAPICategoryFileTimestamp" for file stamp APIs.
	public string AccessedApiType { get; set; }
	// A description?
	public string? Message { get; set; }
	// A url to a longer description?
	public string? Url { get; set; }

	public PrivacyManifestRequirementAttribute (string accessedApiType)
	{
		AccessedApiType = accessedApiType;
	}
}

Then we decorate every API that triggers the privacy manifest requirement with this attribute.

Note that there are APIs in System.Private.CoreLib.dll that triggers the privacy manifest requirement, so this attribute would probably have to be added to System.Private.CoreLib.dll.

If we use such an API internally (either in the runtime or our own code, and the developer can't not use it in any way), we can add the attribute to the assembly in question (either System.Private.CoreLib.dll or Microsoft.iOS.dll).

And finally we handle this attribute in a way similar to how the linker handles trimmability attributes such as the RequiresUnreferencedCodeAttribute: any method that calls another method with this attribute must also have the equivalent attribute, otherwise a warning is raised. For executable projects, the warning can be silenced by adding a privacy manifest to the project with the correct privacy information.

The downside is that this is a significant amount of work upfront, but it pays off once done: there's no manual validation needed once the initial set of APIs have been annotated, the compiler/code analyzers will yell if the attribute isn't forwarded where it needs to be. If Apple adds more APIs to their list of special APIs, the process is very straight forward on our side: we annotate the corresponding APIs, and then, once again, we'll get warnings everywhere we need to forward the attribute.

If we decided to look into this, we should talk with the linker team for any input from them.

[ivanpovazan]

If we wanted to implement this well, here's an idea:

This sounds like a good and extensible approach and I think we should definitely aim our implementation in that direction.

However, as you rightfully pointed out, I also feel it would take a lot of time/work before we get there. Especially if we consider, acceptance on the dotnet/runtime side, integration with the linker, Mono/NativeAOT toolchain, the build system etc. For these reasons, I think this sounds like a solid long-term plan.

Since Apple announced that the privacy manifests will be checked starting on Spring 2024, I think we should focus on a short-term/backup plan which would be a start-in-the-right-direction, but at the same time make sure that our SDKs/apps are compliant in the timely manner.

Therefore, I think as one of the first things to agree upon (which we can utilize in the future) is to define a format which Xamarin bundler would use when producing the xcprivacy files.

For example, one idea for the short-term solution would be that we (on dotnet/runtime side) manually scan and flag all sensitive API's referenced by our frameworks and runtimes (yes, for now this would be a manual process and it would include all our API's even those not referenced by the project). We create a .targets file with a static ItemGroup in an agreed format, which we can deploy in our runtime packs that will be consumed during the build (like Xamarin SDK imports Microsoft.NET.Runtime.MonoTargets.Sdk targets through the workloads). Xamarin SDK and the bundler can then extend this ItemGroup with Xamarin-internal flagged API's and use it in the bundler to produce the required xcprivacy file.

The static list of privacy accessed APIs and the MSBuild target could be something like this:

  <Target Name="CollectPrivacyRequirements" Condition="'$(_IsPublishing)' == 'true'" AfterTargets="<TBD>">
    <ItemGroup>
        <NSPrivacyAccessedAPITypes Include="NSPrivacyAccessedAPICategoryFileTimestamp">
            <NSPrivacyAccessedAPITypeReasons>0A2A.1</NSPrivacyAccessedAPITypeReasons>
            <NSPrivacyAccessedAPIDescription>Used by C# timestamp APIs</NSPrivacyAccessedAPIDescription>
        </NSPrivacyAccessedAPITypes>
        <NSPrivacyAccessedAPITypes Include="NSPrivacyAccessedAPICategoryFileTimestamp">
            <NSPrivacyAccessedAPITypeReasons>C617.1</NSPrivacyAccessedAPITypeReasons>
            <NSPrivacyAccessedAPIDescription>Used to check files</NSPrivacyAccessedAPIDescription>
        </NSPrivacyAccessedAPITypes>
        ...
    </ItemGroup>
  </Target>

NOTE: As an example I used some information Unity includes in their documentation.

This way our users (and other frameworks MAUI?) could also plug-in and extend the ItemGroup for their purposes.

Finally, as a follow-up we would replace this manual process with something more sophisticated and automatic that is compatible with our toolchains similar to what you proposed, but with the same "back-end" that produces the xcprivacy files.

[vitek-karas]

I absolutely agree with Ivan that it seems we need a short-term solution which is reasonably easy to implement.
I also like the attributes as a longer term solution. The trimmer/analyzer/aot might gain a capability to represent the attribute "soon". The design for the more generalized feature attributes seems to allow defining an attribute very similar to this. dotnet/designs#305. I know the Sven is already working on an implementation of this for the analyzer.

[vitek-karas]

Actually, the analyzer change has already been merged: dotnet/runtime#94944

[dalexsoto]

If we wanted to implement this well, here's an idea:

This sounds like a good and extensible approach and I think we should definitely aim our implementation in that direction.

However, as you rightfully pointed out, I also feel it would take a lot of time/work before we get there. Especially if we consider, acceptance on the dotnet/runtime side, integration with the linker, Mono/NativeAOT toolchain, the build system etc. For these reasons, I think this sounds like a solid long-term plan.

Since Apple announced that the privacy manifests will be checked starting on Spring 2024, I think we should focus on a short-term/backup plan which would be a start-in-the-right-direction, but at the same time make sure that our SDKs/apps are compliant in the timely manner.

Therefore, I think as one of the first things to agree upon (which we can utilize in the future) is to define a format which Xamarin bundler would use when producing the xcprivacy files.

For example, one idea for the short-term solution would be that we (on dotnet/runtime side) manually scan and flag all sensitive API's referenced by our frameworks and runtimes (yes, for now this would be a manual process and it would include all our API's even those not referenced by the project). We create a .targets file with a static ItemGroup in an agreed format, which we can deploy in our runtime packs that will be consumed during the build (like Xamarin SDK imports Microsoft.NET.Runtime.MonoTargets.Sdk targets through the workloads). Xamarin SDK and the bundler can then extend this ItemGroup with Xamarin-internal flagged API's and use it in the bundler to produce the required xcprivacy file.

The static list of privacy accessed APIs and the MSBuild target could be something like this:

  <Target Name="CollectPrivacyRequirements" Condition="'$(_IsPublishing)' == 'true'" AfterTargets="<TBD>">
    <ItemGroup>
        <NSPrivacyAccessedAPITypes Include="NSPrivacyAccessedAPICategoryFileTimestamp">
            <NSPrivacyAccessedAPITypeReasons>0A2A.1</NSPrivacyAccessedAPITypeReasons>
            <NSPrivacyAccessedAPIDescription>Used by C# timestamp APIs</NSPrivacyAccessedAPIDescription>
        </NSPrivacyAccessedAPITypes>
        <NSPrivacyAccessedAPITypes Include="NSPrivacyAccessedAPICategoryFileTimestamp">
            <NSPrivacyAccessedAPITypeReasons>C617.1</NSPrivacyAccessedAPITypeReasons>
            <NSPrivacyAccessedAPIDescription>Used to check files</NSPrivacyAccessedAPIDescription>
        </NSPrivacyAccessedAPITypes>
        ...
    </ItemGroup>
  </Target>

NOTE: As an example I used some information Unity includes in their documentation.

This way our users (and other frameworks MAUI?) could also plug-in and extend the ItemGroup for their purposes.

Finally, as a follow-up we would replace this manual process with something more sophisticated and automatic that is compatible with our toolchains similar to what you proposed, but with the same "back-end" that produces the xcprivacy files.

What happens if the linker removes all of the usages of said APIs and there is no need for a privacy manifest? Could we make this more opt-in? For example document all the runtime/macios/MAUI usages and then let the final user fill in the above proposed item groups on their own so we can generate the main bundled privacy manifest.

Then once we have the more advanced annotations we can have the analyzer or linker step to produce this automatically in the future.

I absolutely agree with Ivan that it seems we need a short-term solution which is reasonably easy to implement. I also like the attributes as a longer term solution. The trimmer/analyzer/aot might gain a capability to represent the attribute "soon". The design for the more generalized feature attributes seems to allow defining an attribute very similar to this. dotnet/designs#305. I know the Sven is already working on an implementation of this for the analyzer.

I agree, we need a short term solution now if the long term one is not ready to be implemented on the runtime side.

[vitek-karas]

What happens if the linker removes all of the usages of said APIs and there is no need for a privacy manifest? Could we make this more opt-in? For example document all the runtime/macios/MAUI usages and then let the final user fill in the above proposed item groups on their own so we can generate the main bundled privacy manifest.

I think without some more involved changes (attributes, tooling support, ...) it will be hard to detect the need for this reliably. So I would probably go with the "Unity" approach - add the stuff which runtime may need always. If developers don't want it for some reason, it's a single line of MSBuild to remove everything we put there by default anyway.

In fact for the first version of this, we might go with just some docs, where we provide the snippets for the runtime and the frameworks and so on, ant let it up to the developer to construct the final XML.

For the long term solution - just a question:

• Do we need to propagate the attributes?
• The usefulness of propagation comes from the ability to let the library/app developer know, that there's certain compatibility requirements for using certain API.
• So the question is - do we think developers should be aware of all APIs (.NET) which will add stuff to the privacy manifest - and are they informed enough to make the right call?
• Alternative would be to have the attribute but to not propagate it. We would simply annotate the low-level methods with it and use it generate the private manifest after trimming. Basically have the attribute as part of the internal tooling to generate the manifest.

[rolfbjarne]

• So the question is - do we think developers should be aware of all APIs (.NET) which will add stuff to the privacy manifest - and are they informed enough to make the right call?

It's the developers who have to create the privacy manifest. Only they know why they're using a particular API, and they have to explain the reason in the privacy manifest. All we can do is say "hey, if you use this managed API, then you're effectively using this other native API, which means you'll then you'll have to explain why in the privacy manifest".

At most we can create a privacy manifest with a placeholder reason - but the developers still have to review why they're using the APIs in question.

I agree, we need a short term solution now if the long term one is not ready to be implemented on the runtime side.

We might want to wait until Xcode 16 in June to see what else (if anything) Apple comes up with until we start looking at the long term solution, just in case they throw the whole thing out and do something completely different.

[vitek-karas]

Thanks a lot @rolfbjarne for explaining this to me.
Given that, the "propagated attribute" solution seems like a good idea. We might need one more capability from the analyzers/IDE/compilers - the attribute related diagnostics (warnings) should really only kick in for apps targeting iOS. I don't know if the current feature switch design has the capability to do this. But I also don't think it would be a difficult thing to add.

[rolfbjarne]

We might need one more capability from the analyzers/IDE/compilers - the attribute related diagnostics (warnings) should really only kick in for apps targeting iOS.

Couldn't that be accomplished by only adding the attribute in the implementation assembly for iOS? That way other platforms wouldn't even see the attribute.

[jstedfast]

I got tagged on this for the IDE-side of things. From what I can tell, the IDE won't really need anything because this is all just a plist. Correct?

Maybe, at most, we could maybe add a schema file for the plist editor that we already have?

[vitek-karas]

Couldn't that be accomplished by only adding the attribute in the implementation assembly for iOS? That way other platforms wouldn't even see the attribute.

That would work for trimmer/NativeAOT, but not for the analyzer. The analyzer (since it's in Roslyn) sees reference assemblies, and if I understand it correctly, we have only one set of reference assemblies, they are not RID specific.

[thefex]

Is not that just a plist file inside bundled release IPA ? That way, you could add a custom msbuild step that copies that from project into IPA after build/package step as a temporary workaround + short tutorial/guide how to create it via XCode. You could think and polish better solution for net8/net9+ then - more time to do so for you.

That could give a support for old xamarin workflows (just copy-paste build-step into csproj) as well (XCode 15 support been added to give more time for migration which is a total pain even for non-forms apps)

[FM1973]

Please keep in mind, that a solution for this might be needed for .net 7 as well. We have an app publishes using ,net 7 and did not upgrade to .net 8 until now because of known bugs in .net maui 8. Thanks! Apple will check this starting 1st May 2024.

[durandt]

First warnings ( ITMS-91053 , for referencing) have started rolling in (see dotnet/maui#21296).

Thank you @rolfbjarne for pointing me toward this issue!

I have read the PR #20292 and you guys are really making a great job with platform-specific documentation (kudos @mcumming ). You are very often filling the holes in Google's/Apple's docs. Thank you for that!

In the PR, I found the answers both regarding since when the warnings started to be sent (March 13th) and which warnings are caused by MAUI/Xamarin-macios (I myself am not directly using any of the APIs from the warnings).

Quoting Rolf

It's the developers who have to create the privacy manifest. Only they know why they're using a particular API, and they have to explain the reason in the privacy manifest.

Not so easy really, I am glad I haven't pulled too many 3rd parties to my app...

[jfversluis]

@FM1973 correct, this is needed for any app that you want to release or update in the Apple App Store after May 1, 2024, regardless of what technology or what version of a technology it is built with. Luckily no new functionality has to be implemented for this. As long as you can figure out what the PrivacyInfo.xcprivacy file should look like, add that to your project and you should be good.

The mechanisms to add a file in your app bundle is already working and implemented for a long time, so that should not be the issue.

[LennoxP90]

what do us legacy Xamarin users do to our projects do we have to manually create this privacy file for example my email contained this

ITMS-91053: Missing API declaration - Your app’s code in the “<REDACTED>” file references one or more APIs that require reasons, including the following API categories: NSPrivacyAccessedAPICategoryDiskSpace. While no action is required at this time, starting May 1, 2024, when you upload a new app or app update, you must include a NSPrivacyAccessedAPITypes array in your app’s privacy manifest to provide approved reasons for these APIs used by your app’s code. For more details about this policy, including a list of required reason APIs and approved reasons for usage, visit: https://developer.apple.com/documentation/bundleresources/privacy_manifest_files/describing_use_of_required_reason_api.

ITMS-91053: Missing API declaration - Your app’s code in the “<REDACTED>” file references one or more APIs that require reasons, including the following API categories: NSPrivacyAccessedAPICategorySystemBootTime. While no action is required at this time, starting May 1, 2024, when you upload a new app or app update, you must include a NSPrivacyAccessedAPITypes array in your app’s privacy manifest to provide approved reasons for these APIs used by your app’s code. For more details about this policy, including a list of required reason APIs and approved reasons for usage, visit: https://developer.apple.com/documentation/bundleresources/privacy_manifest_files/describing_use_of_required_reason_api.

ITMS-91053: Missing API declaration - Your app’s code in the “<REDACTED>” file references one or more APIs that require reasons, including the following API categories: NSPrivacyAccessedAPICategoryUserDefaults. While no action is required at this time, starting May 1, 2024, when you upload a new app or app update, you must include a NSPrivacyAccessedAPITypes array in your app’s privacy manifest to provide approved reasons for these APIs used by your app’s code. For more details about this policy, including a list of required reason APIs and approved reasons for usage, visit: https://developer.apple.com/documentation/bundleresources/privacy_manifest_files/describing_use_of_required_reason_api.

ITMS-91053: Missing API declaration - Your app’s code in the “<REDACTED>” file references one or more APIs that require reasons, including the following API categories: NSPrivacyAccessedAPICategoryFileTimestamp. While no action is required at this time, starting May 1, 2024, when you upload a new app or app update, you must include a NSPrivacyAccessedAPITypes array in your app’s privacy manifest to provide approved reasons for these APIs used by your app’s code. For more details about this policy, including a list of required reason APIs and approved reasons for usage, visit: https://developer.apple.com/documentation/bundleresources/privacy_manifest_files/describing_use_of_required_reason_api.

[jfversluis]

@LennoxP90 yes, the same applies to legacy Xamarin (or any iOS app), add that file with the right contents to the root of your app bundle and you're good to go. You do want to understand why these entries are there obviously so you can explain that to your users, but in this case this seems to be the exact APIs that are used by the .NET runtime and Xamarin(.Forms/.Essentials) so that checks out.

[Gekidoku]

@FM1973 correct, this is needed for any app that you want to release or update in the Apple App Store after May 1, 2024, regardless of what technology or what version of a technology it is built with. Luckily no new functionality has to be implemented for this. As long as you can figure out what the PrivacyInfo.xcprivacy file should look like, add that to your project and you should be good.

The mechanisms to add a file in your app bundle is already working and implemented for a long time, so that should not be the issue.

Do we have a template for people to use if the only APIs you use are the ones used by the .net runtime?

[jfversluis]

@Gekidoku I just opened a PR to add the minimal needed version to the .NET MAUI templates, find that here: dotnet/maui#21350

If you're using .NET for iOS without MAUI you can leave out the NSPrivacyAccessedAPICategoryUserDefaults entry.

[tipa]

@mcumming I believe there is a typo here (it should be CA92.1 instead of CAS92.1):

xamarin-macios/docs/apple-privacy-manifest.md

Line 65 in 4854f70

Additionally, if you use the `NSUserDefaults` APIs in your app, you will need to add the `NSPrivacyAccessedAPICategoryUserDefaults` API category, with a reason code of `CAS92.1`.

I also pointed it out in the PR, but maybe it was overlooked.

[jstedfast]

@jfversluis would it make sense to have a FileTemplate for the PrivacyInfo.xcprivacy that has more-or-less all of the key/value pairs that a basic iOS app would hit? I.e. similar to what you are adding for MAUI minus the NSPrivacyAccessedAPICategoryUserDefaults key?

[Gekidoku]

@mcumming I believe there is a typo here (it should be CA92.1 instead of CAS92.1):

xamarin-macios/docs/apple-privacy-manifest.md

Line 65 in 4854f70

Additionally, if you use the `NSUserDefaults` APIs in your app, you will need to add the `NSPrivacyAccessedAPICategoryUserDefaults` API category, with a reason code of `CAS92.1`.

I also pointed it out in the PR, but maybe it was overlooked.

Here (search for NSPrivacyAccessedAPICategoryUserDefaults) i see them list it as CA92.1

so is this apple making a typo in their documentation

read versluis's comment below. ty for the quick work jversluis

[jfversluis]

@tipa @Gekidoku that typo should be fixed by #20352 is there still something wrong?

@jstedfast oh like a FileTemplate in Visual Studio that people could add? Great idea, but my knowledge of the templating engine is limited. I think I wouldn't be able to add an entry to the csproj? If I can't do that with the template I'd say its value is limited. People will still have to go out and find out how to make the privacy manifest file show up in the right place. If they're going to do that, the extra step to paste in the contents isn't that big?

Maybe I'm not understanding you correctly or hopefully this is possible with templates. Feel free to reach out offline and discuss a bit more!

[jstedfast]

The dotnet engine already has FileTemplates that get added to the .csproj when a user selects them in the "Add New Item" dialog box, so it's definitely possible.

[AlleSchonWeg]

Hi,
is this also working with app extensions? Do we need a separate manifest file for the extension?
Reference: https://forums.developer.apple.com/forums/thread/748947

[jfversluis]

@AlleSchonWeg whenever you have the file with the correct contents it should work with whatever Apple decides. As mentioned in the linked forum post as well, here it mentions:

App extensions don’t include privacy information files. The operating system and App Store Connect use the privacy information file in the extension’s host app bundle, in combination with those from third-party SDKs your app links to.

That is also all the information we have at this moment, and if I were to believe this, it should just work whenever you add the correct privacy manifest to your host app, in this case your .NET for iOS or .NET MAUI app.