macOS Mojave and Catalina TCC trigger

[pnerum]

Description

When launching Processing.app and a user wants to use audio input for example, the app doesn’t aks for permission to use the microphone

Expected Behavior

Trigger a window asking the user for the processing.app to access the microphone

Current Behavior

No tcc triggers, so there are no audio or video inputs available

Steps to Reproduce

1. Open Processing.app
2. Import sound library
3. Open examples -->Libraries-->Sound-->IO-->AudioInput

Your Environment

• Processing version: 3.5.4 and 4.0 beta
• Operating System and OS version: 10.14.6

Possible Causes / Solutions

Use the Apple provided API call to add some tags in the Info.plist file.
https://developer.apple.com/documentation/avfoundation/cameras_and_media_capture/requesting_authorization_for_media_capture_on_macos

[pnerum]

I have a Developer ID myself and could perform some more tests.
In the Processing.app bundle you added the "jdk1.8.0_202.jdk”. This bundle has to have the same changes as noted in my first request. I edited the "Processing.app/Contents/PlugIns/jdk1.8.0_202.jdk/Contents/Info.plist” to add these two keys:
<key>NSCameraUsageDescription</key>
<string>Processing Java needs access to the camera for your video calls</string>
<key>NSMicrophoneUsageDescription</key>
<string>Processing Java needs access to the microphone for your calls</string>
Resigned that Bundle with my own Developer ID and it worked!

[pnerum]

[kevinstadler]

@pnerum does the workaround described in #30 work for you?

In order to fix this in the library itself (without requiring the entire Processing app to ask for permissions on startup which it might not end up using) we'd need to ask for the permission programmatically near the code block below, where there's currently only an if statement handling audio permissions on Android. @pnerum since I can't test this myself, can you tell me if:

1. it is possible to request the audio-in permission programmatically only when it's actually needed during runtime?
2. how can we detect reliably whether a TCC trigger is necessary? In the code below it's the class type of the audio manager that tells us we need to test for permission, but maybe in this case a call to read out some system/OS constants will be necessary, do you have a hunch which?
   

   processing-sound/src/processing/sound/AudioIn.java

   Lines 44 to 58 in 7a56aa4

   	if (Engine.getAudioManager() instanceof JSynAndroidAudioDeviceManager) {
   	if (in != 0) {
   	Engine.printWarning("if you want to capture audio from somewhere other than the default\n" +
   	"device on Android, use: new Sound(this).inputDevice(deviceID)\n" +
   	"where for deviceID you can fill in any of Android's MediaRecorder.AudioSource constants.");
   	}
   	// we're on Android, check if the sketch has permission to capture Audio
   	if (!parent.hasPermission(Manifest.permission.RECORD_AUDIO)) {
   	Engine.printError(AudioIn.ANDROID_PERMISSION_WARNING_MESSAGE);
   	throw new AndroidPermissionException("RECORD_AUDIO permission not granted");
   	}
   	Engine.printMessage("capturing audio in from device " + Engine.getSelectedInputDeviceName());
   	}

[pnerum]

Hi Kevin,
In my testing it was only asking for permission when I started the AudioInput application (in runtime) and not by starting Processing.app That’s why Processing.app/Contents/PlugIns/jdk1.8.0_202.jdk/Contents/Info.plist needs the plist to be edited and signed again. It’s the java binary in Processing.app/Contents/PlugIns/jdk1.8.0_202.jdk/Contents/Home/jre/bin/java that’s asking for permission to use the microphone.

As you can see in the logstream output:

2020-01-27 11:13:02.659628+0100 0x1280245 Info 0xe96bc 281 0 tccd: [com.apple.TCC:access] AttributionChain: RESP:{ID: ??, PID[75127], auid: 501, euid: 501, responsible path: '/Users/testing/Desktop/Processing.app/Contents/MacOS/Processing', binary path: '/Users/testing/Desktop/Processing.app/Contents/MacOS/Processing'}, ACC:{ID: net.java.openjdk.cmd, PID[75139], auid: 501, euid: 501, binary path: '/Users/testing/Desktop/Processing.app/Contents/PlugIns/jdk1.8.0_202.jdk/Contents/Home/jre/bin/java'}, REQ:{ID: com.apple.audio.coreaudiod, PID[170], auid: 202, euid: 202, binary path: '/usr/sbin/coreaudiod'}

[kevinstadler]

Oh great! Could you do me a favour and install Minim (an alternative sound library) from within Processing and see if it behaves in the same way when you run its audio-in example?

If the behaviour is the same and the extra permission entry doesn't interfere with Processing startup until a sketch actually attempts to access the microphone then @benfry I think this should be fixed at the level of the Processing app itself?

I don't know how exactly the whole .App bundling thing works but maybe the permissions need to be added here? https://github.com/processing/processing/blob/349f413a3fb63a75e0b096097a5b0ba7f5565198/java/application/Info.plist.tmpl

[pnerum]

Just tested it and Minim does indeed behaves exactly the same as the audioin example.

[pnerum]

Pay attention as the text string of my example should say something more logical. I copied those strings from Skype.app, just for testing purposes.

<string>Processing Java needs access to the camera for your video calls</string>
<string>Processing Java needs access to the microphone for your calls</string>

[kevinstadler]

@pnerum thanks so much, that's a huge help!

@benfry could this be moved back to (and addressed in) the main processing repository?

[pnerum]

I’ve added the examples: https://gist.github.com/pnerum/75a7e56edf545cb3114d5794a53b8699.js

[pnerum]

@kevinstadler No problem! If this gets solved our students and staff will be very happy!

[pnerum]

@kevinstadler and @benfry isn’t this issue in the wrong project?
The TCC trigger also needs to be triggered for video. I just performed a test as you can see in my screenshot.

[pnerum]

Hi @kevinstadler the issue isn’t solved until someone from the developers need to edit the Info.plist and redistribute this updated app version as a new release.
See: #51 (comment)

[pnerum]

@benfry Can you please update the Info.plist so TCC triggers are activated when using audio or video and sign the updated application?
According to the latest version you’ve signed the Processing.app: Authority=Developer ID Application: Ben Fry LLC (8SBRM6J77J)

This needs to be added to Info.plist:

<key>NSCameraUsageDescription</key>
<string>Processing needs access to the camera for your video calls</string>
<key>NSMicrophoneUsageDescription</key>
<string>Processing needs access to the microphone for your calls</string>

[evolutie]

I really hope this gets solved quickly, as our students can't work with camera and microphone until this is solved!

[NoBodyButMe]

Just the same problem over here. I used to work with Processing (camera and mic) for my colleges at UCLL, but this isn't going well anymore.

[dvlahos]

Hope this is fixed soon. Has this also been been added to the known issues list for Processing 4?

[kevinstadler]

@dvlahos not yet as far as I know, could you please add a ticket there to raise awareness

[jmiguelff]

Is there a Info.plist file that I could use to be able to use camera and microphone on macOS Catalina?

Or do we need to wait for a release?

[pnerum]

@jmiguelff You can add it yourself, see: #51 (comment). After you’ve added it you need to sign the app with your own Apple developer ID if you have one.

[chi11i]

I found a workaround until a release for anyone like myself who isn't able to update/sign the .plist.

Open the Processing 3 app through Terminal by typing:
/Applications/Processing.app/Contents/MacOS/Processing;

Then open a sketch that requires use of the mic/camera in the Processing PDE (this doesn't work if you're using Sublime etc as an editor). Then you will get the pop-up that asks if you want to allow Processing to access the mic/camera.

Once you've done it once, the privacy setting seems to stay 'allowed' and you don't need to open Processing through the Terminal again. You can check by looking at your 'security & privacy' settings under 'system preferences'.

Hope that helps others until the fix is sorted.

[RomainAl]

I found a workaround until a release for anyone like myself who isn't able to update/sign the .plist.

Open the Processing 3 app through Terminal by typing:
/Applications/Processing.app/Contents/MacOS/Processing;

Then open a sketch that requires use of the mic/camera in the Processing PDE (this doesn't work if you're using Sublime etc as an editor). Then you will get the pop-up that asks if you want to allow Processing to access the mic/camera.

Once you've done it once, the privacy setting seems to stay 'allowed' and you don't need to open Processing through the Terminal again. You can check by looking at your 'security & privacy' settings under 'system preferences'.

Hope that helps others until the fix is sorted.

THX but it seems only work for sounds ... not for camera ...

[chi11i]

Hey @RomainAl - I'm so not an expert here. But happy to share with you the sketch I used that triggered the video pop-up for me. Its code from a project with Joshua Davis. Email in bio.

[crcdng]

My step-by-step workaround for camera and microphone (requires that you are a member of the Apple Developer program) here:
https://gist.github.com/i3games/b063987dfb62baf5d0afda422631b480

[makakken]

@benfry is it possible tu update with the i3games (#51 (comment)) fix? it's been a while since i was forced to use openFrameworks on macos, and i would preferably return to processing ;)

[mgorbet]

Hi folks. I seem to have at some point gotten my version of Processing to ask for mic access, but I generally use vscode and the processing-java command-line to launch projects. Unfortunately, whatever's going on under the hood there (looks like an enormous java call when I do a ps -eaf command) doesn't trigger the TCC. I'm guessing I now need to give "java" permission?

Any guesses or thoughts on how to trick raw "java" into asking for mic permissions, so I can run Processing sketches from vscode and get on with my projects?

[benfry]

Hi all, sorry that this has been broken for so long. The video and sound libraries were moved out of the core because 1) they were very large, and 2) so that others could maintain them. Unfortunately there's nobody doing that maintenance at the moment, so once again it falls to me. We're all volunteers, making this available for free, and have very limited time (none, in the midst of a pandemic) to work on it.

I've made the necessary changes so that the prompt will show up on in the next 4.0 release (alpha 3), or you can build that code and run it yourself from that repo: https://github.com/processing/processing4