Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Android] Store image/audio/video in FileProvider due to Android 11 updates #215

Closed
wants to merge 9 commits into from
Closed

[Android] Store image/audio/video in FileProvider due to Android 11 updates #215

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
b75c66d
Save image/audio/video to FileProvider instead of MediaStore external…
chriskhongqarma Apr 13, 2021
b8fb20d
Android add provider xml path
chriskhongqarma Apr 14, 2021
f7c2230
use androidx.core.content.FileProvider
chriskhongqarma Apr 14, 2021
54d2652
use android.support.v4.content.FileProvider instead of androidx
chriskhongqarma Apr 14, 2021
b25e2f3
add android support v4 config
chriskhongqarma Apr 14, 2021
bf28483
use androidx framework
chriskhongqarma Apr 14, 2021
0473bb7
enable AndroidX
chriskhongqarma Apr 15, 2021
51392e8
using Android Support Library instead of AndroidX for now, will updat…
chriskhongqarma Apr 19, 2021
937b084
update cordova-plugin-file to newer version
chriskhongqarma Jun 16, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 18 additions & 1 deletion plugin.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ xmlns:android="http://schemas.android.com/apk/res/android"
<engine name="cordova-android" version=">=6.3.0" />
</engines>

<dependency id="cordova-plugin-file" version="^6.0.0" />
<dependency id="cordova-plugin-file" version="^7.0.0" />
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Revert this, master branch has newer dependency requirements


<js-module src="www/CaptureAudioOptions.js" name="CaptureAudioOptions">
<clobbers target="CaptureAudioOptions" />
Expand Down Expand Up @@ -76,6 +76,18 @@ xmlns:android="http://schemas.android.com/apk/res/android"
</feature>
</config-file>

<config-file target="AndroidManifest.xml" parent="application">
<provider
android:name="org.apache.cordova.mediacapture.FileProvider"
android:authorities="${applicationId}.cordova.plugin.mediacapture.provider"
android:exported="false"
android:grantUriPermissions="true" >
<meta-data
android:name="android.support.FILE_PROVIDER_PATHS"
android:resource="@xml/mediacapture_provider_paths"/>
</provider>
</config-file>

<config-file target="AndroidManifest.xml" parent="/*">
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
Expand All @@ -85,10 +97,15 @@ xmlns:android="http://schemas.android.com/apk/res/android"
<source-file src="src/android/Capture.java" target-dir="src/org/apache/cordova/mediacapture" />
<source-file src="src/android/FileHelper.java" target-dir="src/org/apache/cordova/mediacapture" />
<source-file src="src/android/PendingRequests.java" target-dir="src/org/apache/cordova/mediacapture" />
<source-file src="src/android/FileProvider.java" target-dir="src/org/apache/cordova/mediacapture" />
<source-file src="src/android/xml/mediacapture_provider_paths.xml" target-dir="res/xml" />

<js-module src="www/android/init.js" name="init">
<runs />
</js-module>
<preference name="ANDROID_SUPPORT_V4_VERSION" default="27.+"/>
<framework src="com.android.support:support-v4:$ANDROID_SUPPORT_V4_VERSION"/>

Comment on lines +106 to +108
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove this. We do not support the older Android Support Library anymore

</platform>

<!-- ios -->
Expand Down
147 changes: 111 additions & 36 deletions src/android/Capture.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,9 @@ Licensed to the Apache Software Foundation (ASF) under one
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Field;
import java.lang.reflect.Method;
import java.text.SimpleDateFormat;
import java.util.Arrays;
import java.util.Date;

import android.content.ActivityNotFoundException;
import android.os.Build;
Expand Down Expand Up @@ -57,6 +59,8 @@ Licensed to the Apache Software Foundation (ASF) under one
import android.net.Uri;
import android.os.Environment;
import android.provider.MediaStore;
import android.support.v4.content.FileProvider;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replace this with AndroidX

import org.apache.cordova.BuildHelper;

public class Capture extends CordovaPlugin {

Expand All @@ -83,7 +87,12 @@ public class Capture extends CordovaPlugin {
private final PendingRequests pendingRequests = new PendingRequests();

private int numPics; // Number of pictures before capture activity
private Uri imageUri;
private String audioAbsolutePath;
private String imageAbsolutePath;
private String videoAbsolutePath;

private String applicationId;


// public void setContext(Context mCtx)
// {
Expand Down Expand Up @@ -122,6 +131,9 @@ protected void pluginInitialize() {

@Override
public boolean execute(String action, JSONArray args, CallbackContext callbackContext) throws JSONException {
this.applicationId = (String) BuildHelper.getBuildConfigValue(this.cordova.getActivity(), "APPLICATION_ID");
this.applicationId = preferences.getString("applicationId", this.applicationId);

if (action.equals("getFormatData")) {
JSONObject obj = getFormatData(args.getString(0), args.getString(1));
callbackContext.success(obj);
Expand Down Expand Up @@ -234,6 +246,17 @@ private void captureAudio(Request req) {
try {
Intent intent = new Intent(android.provider.MediaStore.Audio.Media.RECORD_SOUND_ACTION);

String timeStamp = new SimpleDateFormat("yyyyMMddHHmmssSSS").format(new Date());
String fileName = "AUDIO_" + timeStamp + ".wav";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
String fileName = "AUDIO_" + timeStamp + ".wav";
String fileName = "cdv_media_capture_audio_" + timeStamp + ".m4a";

When I use the recording software, I get m4a files. I dont suggest changing format.

File audio = new File(getTempDirectoryPath(), fileName);
Copy link
Contributor

@ath0mas ath0mas Nov 6, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(for all 3 capture types)

I would drop the use of getTempDirectoryPath to not write in app temp dir, because this path is cleaned up by the system while such captured audio/image/video should be available afterward in the device media dirs (correct Environment.DIRECTORY_..) as a common practice.

See the Taking photos doc : https://developer.android.com/training/camera/photobasics#TaskPath, really similar to what you have done

  • use File.createTempFile
  • with specific values and correct Environment.DIRECTORY_.. for each of our 3 types
  • and possibly more <external-files-path entries inside mediacapture_provider_paths.xml to match that

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Android training link I shared gives example of using getExternalFilesDir (with corresponding paths.xml) but this writes to the app private dirs and thus requires to "Add the photo to a gallery" to be similar to the auto-indexing of previous Android and iOS mediacapture code!

Note: If you saved your photo to the directory provided by getExternalFilesDir(), the media scanner cannot access the files because they are private to your app.

Do you agree? Or to which dirs will we agree to write by default here?

Any way to capture and write all 3 types to public dirs?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://developer.android.com/guide/topics/media/camera.html#saving-media
gives example of mediaStorageDir defined using Environment.getExternalStoragePublicDirectory

but deprecated since API 29: what to use? are this SO or other correct? with pre and post Android Q code? through ContentResolver? through ACTION_CREATE_DOCUMENT? ...


Uri audioUri = FileProvider.getUriForFile(this.cordova.getActivity(),
this.applicationId + ".cordova.plugin.mediacapture.provider",
audio);
this.audioAbsolutePath = audio.getAbsolutePath();
intent.putExtra(android.provider.MediaStore.EXTRA_OUTPUT, audioUri);
intent.addFlags(Intent.FLAG_GRANT_WRITE_URI_PERMISSION);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why this intent.addFlags(Intent.FLAG_GRANT_WRITE_URI_PERMISSION);?

LOG.d(LOG_TAG, "Recording an audio and saving to: " + this.audioAbsolutePath);
this.cordova.startActivityForResult((CordovaPlugin) this, intent, req.requestCode);
} catch (ActivityNotFoundException ex) {
pendingRequests.resolveWithFailure(req, createErrorObject(CAPTURE_NOT_SUPPORTED, "No Activity found to handle Audio Capture."));
Expand Down Expand Up @@ -276,13 +299,17 @@ private void captureImage(Request req) {

Intent intent = new Intent(android.provider.MediaStore.ACTION_IMAGE_CAPTURE);

ContentResolver contentResolver = this.cordova.getActivity().getContentResolver();
ContentValues cv = new ContentValues();
cv.put(MediaStore.Images.Media.MIME_TYPE, IMAGE_JPEG);
imageUri = contentResolver.insert(MediaStore.Images.Media.EXTERNAL_CONTENT_URI, cv);
LOG.d(LOG_TAG, "Taking a picture and saving to: " + imageUri.toString());
String timeStamp = new SimpleDateFormat("yyyyMMddHHmmssSSS").format(new Date());
String fileName = "IMG_" + timeStamp + ".jpg";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
String fileName = "IMG_" + timeStamp + ".jpg";
String fileName = "cdv_media_capture_image_" + timeStamp + ".jpg";

File image = new File(getTempDirectoryPath(), fileName);

Uri imageUri = FileProvider.getUriForFile(this.cordova.getActivity(),
this.applicationId + ".cordova.plugin.mediacapture.provider",
image);
this.imageAbsolutePath = image.getAbsolutePath();
intent.putExtra(android.provider.MediaStore.EXTRA_OUTPUT, imageUri);
intent.addFlags(Intent.FLAG_GRANT_WRITE_URI_PERMISSION);
LOG.d(LOG_TAG, "Taking a picture and saving to: " + this.imageAbsolutePath);

this.cordova.startActivityForResult((CordovaPlugin) this, intent, req.requestCode);
}
Expand All @@ -301,6 +328,16 @@ private void captureVideo(Request req) {
PermissionHelper.requestPermission(this, req.requestCode, Manifest.permission.CAMERA);
} else {
Intent intent = new Intent(android.provider.MediaStore.ACTION_VIDEO_CAPTURE);
String timeStamp = new SimpleDateFormat("yyyyMMddHHmmssSSS").format(new Date());
String fileName = "VID_" + timeStamp + ".avi";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest ".mp4" instead of ".avi" as it the file extension I get for video capture with previous code without these FileProvider changes

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it save to assume file extension to begin with? I imagine this could differ depending on the underlying camera app.

Copy link
Contributor

@ath0mas ath0mas Nov 8, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@breautek I think you're right :/

"but" one more link in Android training docs should be interesting to follow : https://developer.android.com/guide/topics/media/camera.html#saving-media
for its getOutputMediaFile(), with output switch for MEDIA_TYPE_IMAGE or MEDIA_TYPE_VIDEO = "IMG_x.jpg" // "VID_x.mp4"

a similar output extension of .mp4 can be found in CameraX training too

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, well if the docs shows an assumption of .mp4 I guess it's fine to assume!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
String fileName = "VID_" + timeStamp + ".avi";
String fileName = "cdv_media_capture_video_" + timeStamp + ".mp4";

File movie = new File(getTempDirectoryPath(), fileName);

Uri videoUri = FileProvider.getUriForFile(this.cordova.getActivity(),
this.applicationId + ".cordova.plugin.mediacapture.provider",
movie);
this.videoAbsolutePath = movie.getAbsolutePath();
intent.putExtra(android.provider.MediaStore.EXTRA_OUTPUT, videoUri);
intent.addFlags(Intent.FLAG_GRANT_WRITE_URI_PERMISSION);

if(Build.VERSION.SDK_INT > 7){
intent.putExtra("android.intent.extra.durationLimit", req.duration);
Expand Down Expand Up @@ -369,10 +406,8 @@ else if (resultCode == Activity.RESULT_CANCELED) {


public void onAudioActivityResult(Request req, Intent intent) {
Copy link
Contributor

@ath0mas ath0mas Nov 8, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can remove the Intent intent param for the 3 onAudio/Image/VideoActivityResult not used anymore

// Get the uri of the audio clip
Uri data = intent.getData();
// create a file object from the uri
req.results.put(createMediaFile(data));
// create a file object from the audio absolute path
req.results.put(createMediaFileWithAbsolutePath(this.audioAbsolutePath));

if (req.results.length() >= req.limit) {
// Send Uri back to JavaScript for listening to audio
Expand All @@ -385,9 +420,7 @@ public void onAudioActivityResult(Request req, Intent intent) {

public void onImageActivityResult(Request req) {
// Add image to results
req.results.put(createMediaFile(imageUri));

checkForDuplicateImage();
req.results.put(createMediaFileWithAbsolutePath(this.imageAbsolutePath));

if (req.results.length() >= req.limit) {
// Send Uri back to JavaScript for viewing image
Expand All @@ -399,32 +432,18 @@ public void onImageActivityResult(Request req) {
}

public void onVideoActivityResult(Request req, Intent intent) {
Uri data = null;

if (intent != null){
// Get the uri of the video clip
data = intent.getData();
}

if( data == null){
File movie = new File(getTempDirectoryPath(), "Capture.avi");
data = Uri.fromFile(movie);
}

// create a file object from the uri
if(data == null) {
if(this.videoAbsolutePath != null) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest to remove the check on videoAbsolutePath != null ; no such check for audio or image, and the absolutePath is always defined when this method is called

req.results.put(createMediaFileWithAbsolutePath(this.videoAbsolutePath));
} else {
pendingRequests.resolveWithFailure(req, createErrorObject(CAPTURE_NO_MEDIA_FILES, "Error: data is null"));
}
else {
req.results.put(createMediaFile(data));

if (req.results.length() >= req.limit) {
// Send Uri back to JavaScript for viewing video
pendingRequests.resolveWithSuccess(req);
} else {
// still need to capture more video clips
captureVideo(req);
}
if (req.results.length() >= req.limit) {
// Send Uri back to JavaScript for viewing video
pendingRequests.resolveWithSuccess(req);
} else {
// still need to capture more videos
captureVideo(req);
}
}

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should createMediaFile(Uri data) be removed? It doesn't appear to be used / replaced with createMediaFileWithAbsolutePath(String path)

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@chriskhongqarma Sorry if this is the incorrect place for this comment.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 as createMediaFile(Uri data) is not used anymore here, fully replaced by createMediaFileWithAbsolutePath(String path)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and other unused elements now like queryImgDB, checkForDuplicateImage, whichContentStore, numPics that should be removed too

Expand Down Expand Up @@ -488,6 +507,62 @@ private JSONObject createMediaFile(Uri data) {
return obj;
}

/**
* Creates a JSONObject that represents a File from the Uri
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

from the absolute path

*
* @param path the absolute path saved in FileProvider of the audio/image/video
* @return a JSONObject that represents a File
* @throws IOException
*/
private JSONObject createMediaFileWithAbsolutePath(String path) {
File fp = new File(path);
JSONObject obj = new JSONObject();

Class webViewClass = webView.getClass();
PluginManager pm = null;
try {
Method gpm = webViewClass.getMethod("getPluginManager");
pm = (PluginManager) gpm.invoke(webView);
} catch (NoSuchMethodException e) {
} catch (IllegalAccessException e) {
} catch (InvocationTargetException e) {
}
if (pm == null) {
try {
Field pmf = webViewClass.getField("pluginManager");
pm = (PluginManager)pmf.get(webView);
} catch (NoSuchFieldException e) {
} catch (IllegalAccessException e) {
}
}
FileUtils filePlugin = (FileUtils) pm.getPlugin("File");
LocalFilesystemURL url = filePlugin.filesystemURLforLocalPath(fp.getAbsolutePath());

try {
// File properties
obj.put("name", fp.getName());
obj.put("fullPath", Uri.fromFile(fp));
if (url != null) {
obj.put("localURL", url.toString());
}
// Because of an issue with MimeTypeMap.getMimeTypeFromExtension() all .3gpp files
// are reported as video/3gpp. I'm doing this hacky check of the URI to see if it
// is stored in the audio or video content store.
if (fp.getAbsoluteFile().toString().endsWith(".3gp") || fp.getAbsoluteFile().toString().endsWith(".3gpp")) {
obj.put("type", VIDEO_3GPP);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why just this line? was previously

                if (data.toString().contains("/audio/")) {
                    obj.put("type", AUDIO_3GPP);
                } else {
                    obj.put("type", VIDEO_3GPP);
                }

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the target file is always defined now with file extension/type? (video always as avi/mp4, and audio as wav) so this block about .3gpp files may be removed?
and only rely on the default obj.put("type", FileHelper.getMimeType(Uri.fromFile(fp), cordova)); ?

} else {
obj.put("type", FileHelper.getMimeType(Uri.fromFile(fp), cordova));
}

obj.put("lastModifiedDate", fp.lastModified());
obj.put("size", fp.length());
} catch (JSONException e) {
// this will never happen
e.printStackTrace();
}
return obj;
}

private JSONObject createErrorObject(int code, String message) {
JSONObject obj = new JSONObject();
try {
Expand Down
21 changes: 21 additions & 0 deletions src/android/FileProvider.java
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
/*
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
*/
package org.apache.cordova.mediacapture;

public class FileProvider extends android.support.v4.content.FileProvider {}
21 changes: 21 additions & 0 deletions src/android/xml/mediacapture_provider_paths.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
<?xml version="1.0" encoding="utf-8"?>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cordova plugins usually store res files into src/android/res: so use src/android/res/xml/mediacapture_provider_paths.xml instead of src/android/xml/mediacapture_provider_paths.xml

<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

<paths xmlns:android="http://schemas.android.com/apk/res/android">
<cache-path name="cache_files" path="." />
</paths>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
</paths>
</paths>

Add blank line