[Android] Store image/audio/video in FileProvider due to Android 11 updates #215

chriskhongqarma · 2021-04-13T10:52:42Z

Android

Motivation and Context

Since the Storage updates in Android 11, it would be impossible for app to get access to files (image, audio, video) saved into MediaStore external storage, as how it is implemented now. Starting from 2021 August, Android apps are required to target Android API level 30 (Android 11), which means it is critical that we need a new way to store files for media-capture plugin. More details are included in the issue #210.
Here is a suggested implementation for the above issue.

Description

The idea is to use a ContentProvider (a FileProvider, in this case) as a common database to store files, so that both our app and other apps can get access to.
An empty file (audio, image, video) with unique file name (which is generated using current timestamp) and corresponding file format will be created before capturing. Unique file name prevents the file from being duplicated. We use FileProvider to create a Uri for the file, and send it through capturing intents as EXTRA_OUTPUT, so that the created file will be saved with the above Uri. Meanwhile, we store the file absolute path as a global variable, so that it can be used when the intent returns results.
As the file is saved into FileProvider, the app will get access and have control over the created file.

Testing

captureAudio, captureImage, captureVideo features are tested over various Android devices (from Android 7 to Android 11 are covered) using browserstack app live. The app then has full access over the generated media file (move, delete, transcode, etc).

Checklist

I've run the tests to see all new and existing tests pass
I added automated test coverage as appropriate for this change
Commit is prefixed with (platform) if this change only applies to one platform (e.g. (android))
If this Pull Request resolves an issue, I linked to the issue in the text above (and used the correct keyword to close issues using keywords)
I've updated the documentation if necessary

… storage, which will not be possible for app to access since Android 11

src/android/Capture.java

src/android/FileProvider.java

breautek · 2021-04-14T13:04:38Z

Cordova-android AndroidX reference: apache/cordova-android#1052

plugin.xml

…e to AndroidX when cordova-android 10 is release, if necessary

andreycruz16 · 2021-04-20T08:02:13Z

Does this fix this issue?
I can't play the recorded video on my WebView.

Cordova: 8.1.2
Cordova-Android: 9.0.0
cordova-plugin-media-capture: 1.4.3

chriskhongqarma · 2021-04-20T11:02:50Z

@andreycruz16 I can see that the recorded video is still saved in external storage. Could you make sure that you pulled the code from this PR? It should save the video into a FileProvider, and the video uri should look like this. file:///data/user/0/{applicationID}/cache/VID_20210420130001621.avi

mirko77 · 2021-04-26T16:37:30Z

@andreycruz16 , @chriskhongqarma On android 11, Lineage OS 18.1, the video gets saved at path:

fullPath: "file:///data/user/0/{applicationID}/cache/Capture.avi"

but I cannot find any file there (using a root explorer), therefore I am getting error 1 (not found) when trying to play it or copy it.

Works fine in older Android versions

Any suggestions?

chriskhongqarma · 2021-04-26T17:44:09Z

@mirko77 As I explained in this PR,
Since the Storage updates in Android 11, it would be impossible for app to get access to files (image, audio, video) saved into MediaStore external storage, as how it is implemented now..
I suggested a solution with this PR and have been waiting for it to be merged. You can pull the code from this PR and try to see if it works. The file recorded should be saved into a path like this:
file:///data/user/0/{applicationID}/cache/VID_20210420130001621.avi, with 20210420130001621 as the current timestamp. Since it is saved into a FileProvider, your app will have full access to the file.

breautek · 2021-04-26T18:04:59Z

I suggested a solution with this PR and have been waiting for it to be merged.

Just FYI: this PR isn't being ignored, just busy preparing cordova-android@10 for API 30 support. I'm personally spending time on weekends trying to get cordova-plugin-file working with raw file paths (so usage stays pretty much the same). Then I'll be focusing on the media/media-capture plugins.

It's my goal to have all of these plugins ready to be released at the same time, or shortly after cordova-android@10 is released... (But I can't make any promises...)

victorvhpg · 2021-06-16T19:18:20Z

@breautek any updates?

PabbleDabble · 2021-07-27T12:40:00Z

src/android/Capture.java

+            pendingRequests.resolveWithSuccess(req);
+        } else {
+            // still need to capture more videos
+            captureVideo(req);
        }
    }



Should createMediaFile(Uri data) be removed? It doesn't appear to be used / replaced with createMediaFileWithAbsolutePath(String path)

@chriskhongqarma Sorry if this is the incorrect place for this comment.

+1 as createMediaFile(Uri data) is not used anymore here, fully replaced by createMediaFileWithAbsolutePath(String path)

and other unused elements now like queryImgDB, checkForDuplicateImage, whichContentStore, numPics that should be removed too

PabbleDabble · 2021-07-27T12:45:25Z

Would it be possible to add search terms / more text to this issue? I spent the better part of a day tracking down an error that appears to be directly addressed by this PR (thank you by the way!!)

In the 3.0.3 code, within createMediaFile(Uri data) after a video capture on Galaxy Tab A7 on Android 11, this line File fp = webView.getResourceApi().mapUriToFile(data); yields null (as the uri is NOT in the local system on this device as you noted in your issue).

Then, later in createMediaFile(Uri data) this line is run LocalFilesystemURL url = filePlugin.filesystemURLforLocalPath(fp.getAbsolutePath()); and since the variable fp is null, you get this error in logcat...

java.lang.NullPointerException: Attempt to invoke virtual method 'java.lang.String java.io.File.getAbsolutePath()' on a null object reference

and the plugin crashes, along with our app.

Request: Could some portion of the logcat error be added to the search terms on this page/issue? Maybe I'm wrong, but this might be a nice way for others who might see this connect that error, with this issue / PR.

Sorry if this is a dumb question. Thank you so much again for this PR!!

davidofwatkins · 2021-10-14T20:51:42Z

Hey folks, now that cordova-android@10 has been out for a few months, are there any updates on this? Google's deadline for existing apps to target API 30 is approaching (November), so this fix is becoming more important.

I gave this a test on a couple Android 10-12 devices and it seemed to work, though I see this isn't using AndroidX yet. Let me know if there's anything I can do to help move this along.

ath0mas · 2021-11-06T01:15:36Z

src/android/Capture.java

+            // are reported as video/3gpp. I'm doing this hacky check of the URI to see if it
+            // is stored in the audio or video content store.
+            if (fp.getAbsoluteFile().toString().endsWith(".3gp") || fp.getAbsoluteFile().toString().endsWith(".3gpp")) {
+                obj.put("type", VIDEO_3GPP);


why just this line? was previously

if (data.toString().contains("/audio/")) { obj.put("type", AUDIO_3GPP); } else { obj.put("type", VIDEO_3GPP); }

the target file is always defined now with file extension/type? (video always as avi/mp4, and audio as wav) so this block about .3gpp files may be removed?
and only rely on the default obj.put("type", FileHelper.getMimeType(Uri.fromFile(fp), cordova)); ?

ath0mas · 2021-11-06T01:19:15Z

src/android/Capture.java

@@ -488,6 +507,62 @@ private JSONObject createMediaFile(Uri data) {
        return obj;
    }

+    /**
+     * Creates a JSONObject that represents a File from the Uri


from the absolute path

ath0mas · 2021-11-06T01:34:29Z

src/android/Capture.java

@@ -234,6 +246,17 @@ private void captureAudio(Request req) {
          try {
              Intent intent = new Intent(android.provider.MediaStore.Audio.Media.RECORD_SOUND_ACTION);

+              String timeStamp = new SimpleDateFormat("yyyyMMddHHmmssSSS").format(new Date());
+              String fileName = "AUDIO_" + timeStamp + ".wav";
+              File audio = new File(getTempDirectoryPath(), fileName);


(for all 3 capture types)

I would drop the use of getTempDirectoryPath to not write in app temp dir, because this path is cleaned up by the system while such captured audio/image/video should be available afterward in the device media dirs (correct Environment.DIRECTORY_..) as a common practice.

See the Taking photos doc : https://developer.android.com/training/camera/photobasics#TaskPath, really similar to what you have done

use File.createTempFile

with specific values and correct Environment.DIRECTORY_.. for each of our 3 types

and possibly more <external-files-path entries inside mediacapture_provider_paths.xml to match that

The Android training link I shared gives example of using getExternalFilesDir (with corresponding paths.xml) but this writes to the app private dirs and thus requires to "Add the photo to a gallery" to be similar to the auto-indexing of previous Android and iOS mediacapture code!

Note: If you saved your photo to the directory provided by getExternalFilesDir(), the media scanner cannot access the files because they are private to your app.

Do you agree? Or to which dirs will we agree to write by default here?

Any way to capture and write all 3 types to public dirs?

https://developer.android.com/guide/topics/media/camera.html#saving-media
gives example of mediaStorageDir defined using Environment.getExternalStoragePublicDirectory

but deprecated since API 29: what to use? are this SO or other correct? with pre and post Android Q code? through ContentResolver? through ACTION_CREATE_DOCUMENT? ...

ath0mas · 2021-11-06T10:08:16Z

@chriskhongqarma should be rebased to resolve conflicts since #192 has been merged into master branch yesterday

ath0mas · 2021-11-08T19:08:39Z

src/android/xml/mediacapture_provider_paths.xml

@@ -0,0 +1,21 @@
+<?xml version="1.0" encoding="utf-8"?>


cordova plugins usually store res files into src/android/res: so use src/android/res/xml/mediacapture_provider_paths.xml instead of src/android/xml/mediacapture_provider_paths.xml

ath0mas · 2021-11-08T19:26:01Z

src/android/Capture.java

+                      audio);
+              this.audioAbsolutePath = audio.getAbsolutePath();
+              intent.putExtra(android.provider.MediaStore.EXTRA_OUTPUT, audioUri);
+              intent.addFlags(Intent.FLAG_GRANT_WRITE_URI_PERMISSION);


why this intent.addFlags(Intent.FLAG_GRANT_WRITE_URI_PERMISSION);?

the Android training https://developer.android.com/training/camera/photobasics#TaskPath does not have this line

seems to run fine without it in my test app

ath0mas · 2021-11-08T19:31:13Z

src/android/Capture.java

@@ -301,6 +328,16 @@ private void captureVideo(Request req) {
            PermissionHelper.requestPermission(this, req.requestCode, Manifest.permission.CAMERA);
        } else {
            Intent intent = new Intent(android.provider.MediaStore.ACTION_VIDEO_CAPTURE);
+            String timeStamp = new SimpleDateFormat("yyyyMMddHHmmssSSS").format(new Date());
+            String fileName = "VID_" + timeStamp + ".avi";


I suggest ".mp4" instead of ".avi" as it the file extension I get for video capture with previous code without these FileProvider changes

Is it save to assume file extension to begin with? I imagine this could differ depending on the underlying camera app.

@breautek I think you're right :/

"but" one more link in Android training docs should be interesting to follow : https://developer.android.com/guide/topics/media/camera.html#saving-media
for its getOutputMediaFile(), with output switch for MEDIA_TYPE_IMAGE or MEDIA_TYPE_VIDEO = "IMG_x.jpg" // "VID_x.mp4"

a similar output extension of .mp4 can be found in CameraX training too

Ah, well if the docs shows an assumption of .mp4 I guess it's fine to assume!

ath0mas · 2021-11-08T19:35:27Z

src/android/Capture.java

-
-        // create a file object from the uri
-        if(data == null) {
+        if(this.videoAbsolutePath != null) {


I suggest to remove the check on videoAbsolutePath != null ; no such check for audio or image, and the absolutePath is always defined when this method is called

ath0mas · 2021-11-08T19:39:29Z

src/android/Capture.java

@@ -369,10 +406,8 @@ else if (resultCode == Activity.RESULT_CANCELED) {


    public void onAudioActivityResult(Request req, Intent intent) {


can remove the Intent intent param for the 3 onAudio/Image/VideoActivityResult not used anymore

chrisjdev · 2022-06-16T17:01:11Z

I tested the 4.0 release, and it still has an error for recording a video. Taking a picture still works. The fork works for both video and pictures.

mirko77 · 2023-06-13T15:35:54Z

@chriskhongqarma
On Android 13 and API 33 (Gradle 8), now that fork throws an error
Couldn't find meta-data for provider with authority null.cordova.plugin.mediacapture.provider

because this.applicationID is null in Capture.java (line 134/135)


this.applicationId = (String) BuildHelper.getBuildConfigValue(this.cordova.getActivity(), "APPLICATION_ID");
this.applicationId = preferences.getString("applicationId", this.applicationId);

this is fixed by using
this.applicationId = cordova.getContext().getPackageName();

which is what the camera plugin is doing as well (@breautek)-> apache/cordova-plugin-camera#827 (review)

shivamsharmanps · 2023-07-13T15:52:37Z

Can this branch use for android 12 13 issue?

mirko77 · 2023-07-13T16:02:14Z

@shivamsharmanps yes but you need to apply the changes I mentioned, or use this fork
https://github.com/epicollect5/cordova-plugin-media-capture

erisu

I suggest rebase against current master branch, resolve conflicts, and apply suggestions that everyone pointed out.

erisu · 2023-08-09T00:03:19Z

plugin.xml

@@ -34,7 +34,7 @@ xmlns:android="http://schemas.android.com/apk/res/android"
        <engine name="cordova-android" version=">=6.3.0" />
    </engines>

-    <dependency id="cordova-plugin-file" version="^6.0.0" />
+    <dependency id="cordova-plugin-file" version="^7.0.0" />


Revert this, master branch has newer dependency requirements

erisu · 2023-08-09T00:03:48Z

plugin.xml

+        <preference name="ANDROID_SUPPORT_V4_VERSION" default="27.+"/>
+        <framework src="com.android.support:support-v4:$ANDROID_SUPPORT_V4_VERSION"/>
+


Remove this. We do not support the older Android Support Library anymore

erisu · 2023-08-09T00:04:25Z

src/android/Capture.java

@@ -57,6 +59,8 @@ Licensed to the Apache Software Foundation (ASF) under one
 import android.net.Uri;
 import android.os.Environment;
 import android.provider.MediaStore;
+import android.support.v4.content.FileProvider;


Replace this with AndroidX

erisu · 2023-08-09T00:05:34Z

src/android/Capture.java

@@ -234,6 +246,17 @@ private void captureAudio(Request req) {
          try {
              Intent intent = new Intent(android.provider.MediaStore.Audio.Media.RECORD_SOUND_ACTION);

+              String timeStamp = new SimpleDateFormat("yyyyMMddHHmmssSSS").format(new Date());
+              String fileName = "AUDIO_" + timeStamp + ".wav";


Suggested change

String fileName = "AUDIO_" + timeStamp + ".wav";

String fileName = "cdv_media_capture_audio_" + timeStamp + ".m4a";

When I use the recording software, I get m4a files. I dont suggest changing format.

erisu · 2023-08-09T00:06:35Z

src/android/Capture.java

-            imageUri = contentResolver.insert(MediaStore.Images.Media.EXTERNAL_CONTENT_URI, cv);
-            LOG.d(LOG_TAG, "Taking a picture and saving to: " + imageUri.toString());
+            String timeStamp = new SimpleDateFormat("yyyyMMddHHmmssSSS").format(new Date());
+            String fileName = "IMG_" + timeStamp + ".jpg";


Suggested change

String fileName = "IMG_" + timeStamp + ".jpg";

String fileName = "cdv_media_capture_image_" + timeStamp + ".jpg";

erisu · 2023-08-09T00:07:24Z

src/android/Capture.java

@@ -301,6 +328,16 @@ private void captureVideo(Request req) {
            PermissionHelper.requestPermission(this, req.requestCode, Manifest.permission.CAMERA);
        } else {
            Intent intent = new Intent(android.provider.MediaStore.ACTION_VIDEO_CAPTURE);
+            String timeStamp = new SimpleDateFormat("yyyyMMddHHmmssSSS").format(new Date());
+            String fileName = "VID_" + timeStamp + ".avi";


Suggested change

String fileName = "VID_" + timeStamp + ".avi";

String fileName = "cdv_media_capture_video_" + timeStamp + ".mp4";

erisu · 2023-08-09T00:08:21Z

src/android/xml/mediacapture_provider_paths.xml

+
+<paths xmlns:android="http://schemas.android.com/apk/res/android">
+    <cache-path name="cache_files" path="." />
+</paths>


Suggested change

</paths>

</paths>

Add blank line

Save image/audio/video to FileProvider instead of MediaStore external…

b75c66d

… storage, which will not be possible for app to access since Android 11

chriskhongqarma changed the title ~~[Android] Save image/audio/video to FileProvider instead of MediaStore external…~~ [Android] Store image/audio/video in FileProvider due to Android 11 updates Apr 13, 2021

chriskhongqarma mentioned this pull request Apr 13, 2021

iOS: save video file to gallery should be optional #216

Open

chriskhongqarma added 4 commits April 14, 2021 09:11

Android add provider xml path

b8fb20d

use androidx.core.content.FileProvider

f7c2230

use android.support.v4.content.FileProvider instead of androidx

54d2652

add android support v4 config

b25e2f3

chriskhongqarma marked this pull request as ready for review April 14, 2021 09:11

breautek reviewed Apr 14, 2021

View reviewed changes

src/android/Capture.java Outdated Show resolved Hide resolved

src/android/FileProvider.java Outdated Show resolved Hide resolved

use androidx framework

bf28483

breautek added this to the 4.0.0 milestone Apr 14, 2021

breautek added enhancement platform: android labels Apr 14, 2021

chriskhongqarma marked this pull request as draft April 14, 2021 17:13

enable AndroidX

0473bb7

chriskhongqarma marked this pull request as ready for review April 15, 2021 08:08

erisu reviewed Apr 15, 2021

View reviewed changes

plugin.xml Outdated Show resolved Hide resolved

using Android Support Library instead of AndroidX for now, will updat…

51392e8

…e to AndroidX when cordova-android 10 is release, if necessary

PabbleDabble mentioned this pull request Jul 26, 2021

File mapUriToFile returns null apache/cordova-android#492

Closed

PabbleDabble reviewed Jul 27, 2021

View reviewed changes

PabbleDabble mentioned this pull request Jul 27, 2021

MediaCapture CaptureVideo immediately closing app on Android 11 #227

Closed

3 tasks

ath0mas mentioned this pull request Nov 6, 2021

fix(android): prevent app crash caused by NPE on intent data or mediaFile #232

Merged

5 tasks

ath0mas reviewed Nov 6, 2021

View reviewed changes

ath0mas reviewed Nov 8, 2021

View reviewed changes

davidofwatkins mentioned this pull request Nov 16, 2021

Fix Scoped Storage (Android API 30 Requirement) zenput/cordova-plugin-media-capture#1

Merged

mirko77 mentioned this pull request Apr 11, 2022

Cannot use in Android 10+ #233

Open

2 tasks

breautek added this to In progress in 4.x via automation Apr 12, 2022

update cordova-plugin-file to newer version

937b084

erisu mentioned this pull request Aug 7, 2023

[PoC] fix(android): API 30+ capture audio #277

Draft

5 tasks

erisu requested changes Aug 9, 2023

View reviewed changes

		@@ -369,10 +406,8 @@ else if (resultCode == Activity.RESULT_CANCELED) {


		public void onAudioActivityResult(Request req, Intent intent) {

		<preference name="ANDROID_SUPPORT_V4_VERSION" default="27.+"/>
		<framework src="com.android.support:support-v4:$ANDROID_SUPPORT_V4_VERSION"/>

	String fileName = "AUDIO_" + timeStamp + ".wav";
	String fileName = "cdv_media_capture_audio_" + timeStamp + ".m4a";

	String fileName = "IMG_" + timeStamp + ".jpg";
	String fileName = "cdv_media_capture_image_" + timeStamp + ".jpg";

	String fileName = "VID_" + timeStamp + ".avi";
	String fileName = "cdv_media_capture_video_" + timeStamp + ".mp4";

[Android] Store image/audio/video in FileProvider due to Android 11 updates #215

Are you sure you want to change the base?

[Android] Store image/audio/video in FileProvider due to Android 11 updates #215

Conversation

chriskhongqarma commented Apr 13, 2021 • edited

Android

Motivation and Context

Description

Testing

Checklist

breautek commented Apr 14, 2021

andreycruz16 commented Apr 20, 2021 • edited

chriskhongqarma commented Apr 20, 2021

mirko77 commented Apr 26, 2021

chriskhongqarma commented Apr 26, 2021 • edited

breautek commented Apr 26, 2021

victorvhpg commented Jun 16, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

PabbleDabble commented Jul 27, 2021 • edited

davidofwatkins commented Oct 14, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ath0mas Nov 6, 2021 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ath0mas commented Nov 6, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ath0mas Nov 8, 2021 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ath0mas Nov 8, 2021 • edited

Choose a reason for hiding this comment

chrisjdev commented Jun 16, 2022

mirko77 commented Jun 13, 2023 • edited

shivamsharmanps commented Jul 13, 2023

mirko77 commented Jul 13, 2023

erisu left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chriskhongqarma commented Apr 13, 2021 •

edited

andreycruz16 commented Apr 20, 2021 •

edited

chriskhongqarma commented Apr 26, 2021 •

edited

PabbleDabble commented Jul 27, 2021 •

edited

ath0mas Nov 6, 2021 •

edited

ath0mas Nov 8, 2021 •

edited

ath0mas Nov 8, 2021 •

edited

mirko77 commented Jun 13, 2023 •

edited