Skip to content

Commit

Permalink
[camerax] Prevent using unsupported concurrent camera use cases (#6608)
Browse files Browse the repository at this point in the history
Adds logic + documentation to prevent the usage of unsupported (by CameraX) concurrent camera use cases.

The CameraX plugin should only support the concurrent camera use cases supported by Camerax; see [their documentation](https://developer.android.com/media/camera/camerax/architecture#combine-use-cases) for more information. To avoid the usage of unsupported concurrent use cases, this PR changes the plugin so that it behaves according to the following:

* If the preview is paused (via `pausePreview`), concurrent video recording and image streaming (via `startVideoCapturing(cameraId, VideoCaptureOptions(streamCallback:...))`) is supported.
* If the preview is not paused
  * **and** the camera device is at least supported hardware [`LIMITED`](https://developer.android.com/reference/android/hardware/camera2/CameraMetadata#INFO_SUPPORTED_HARDWARE_LEVEL_LIMITED), then concurrent
    image capture and video recording is supported.
  * **and** the camera device is at least supported hardware [`LEVEL_3`](https://developer.android.com/reference/android/hardware/camera2/CameraMetadata#INFO_SUPPORTED_HARDWARE_LEVEL_3), then concurrent video recording and image streaming is supported, but if used, concurrent image capture alongside this is not supported.

Also changes `Surface` constants to match Dart style and small documentation nits I noticed.

Fixes flutter/flutter#146395, fixes flutter/flutter#144414.
  • Loading branch information
camsim99 committed May 6, 2024
1 parent 2dfe645 commit 79d733c
Show file tree
Hide file tree
Showing 26 changed files with 2,333 additions and 545 deletions.
4 changes: 4 additions & 0 deletions packages/camera/camera_android_camerax/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
## 0.6.4

* Prevents usage of unsupported concurrent `UseCase`s based on the capabiliites of the camera device.

## 0.6.3

* Shortens default interval that internal Java `InstanceManager` uses to release garbage collected weak references to
Expand Down
19 changes: 19 additions & 0 deletions packages/camera/camera_android_camerax/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,22 @@ from your project's root directory.

## Limitations

### Concurrent preview display, video recording, image capture, and image streaming

The CameraX plugin only supports the concurrent camera use cases supported by Camerax; see
[their documentation][6] for more information. To avoid the usage of unsupported concurrent
use cases, the plugin behaves according to the following:

* If the preview is paused (via `pausePreview`), concurrent video recording and image capture
and/or image streaming (via `startVideoCapturing(cameraId, VideoCaptureOptions(streamCallback:...))`)
is supported.
* If the preview is not paused
* **and** the camera device is at least supported hardware [`LIMITED`][8], then concurrent
image capture and video recording is supported.
* **and** the camera device is at least supported hardware [`LEVEL_3`][7], then concurrent
video recording and image streaming is supported, but concurrent video recording, image
streaming, and image capture is not supported.

### 240p resolution configuration for video recording

240p resolution configuration for video recording is unsupported by CameraX,
Expand All @@ -45,6 +61,9 @@ For more information on contributing to this plugin, see [`CONTRIBUTING.md`](CON
[3]: https://docs.flutter.dev/packages-and-plugins/developing-packages#non-endorsed-federated-plugin
[4]: https://pub.dev/packages/camera_android
[5]: https://github.com/flutter/flutter/issues/new/choose
[6]: https://developer.android.com/media/camera/camerax/architecture#combine-use-cases
[7]: https://developer.android.com/reference/android/hardware/camera2/CameraMetadata#INFO_SUPPORTED_HARDWARE_LEVEL_3
[8]: https://developer.android.com/reference/android/hardware/camera2/CameraMetadata#INFO_SUPPORTED_HARDWARE_LEVEL_LIMITED
[120462]: https://github.com/flutter/flutter/issues/120462
[125915]: https://github.com/flutter/flutter/issues/125915
[120715]: https://github.com/flutter/flutter/issues/120715
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
// Copyright 2013 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

package io.flutter.plugins.camerax;

import androidx.annotation.NonNull;
import androidx.annotation.Nullable;
import androidx.camera.camera2.interop.Camera2CameraInfo;
import io.flutter.plugin.common.BinaryMessenger;
import io.flutter.plugins.camerax.GeneratedCameraXLibrary.Camera2CameraInfoFlutterApi;

public class Camera2CameraInfoFlutterApiImpl extends Camera2CameraInfoFlutterApi {
private final InstanceManager instanceManager;

public Camera2CameraInfoFlutterApiImpl(
@Nullable BinaryMessenger binaryMessenger, @Nullable InstanceManager instanceManager) {
super(binaryMessenger);
this.instanceManager = instanceManager;
}

void create(@NonNull Camera2CameraInfo camera2CameraInfo, @Nullable Reply<Void> reply) {
if (!instanceManager.containsInstance(camera2CameraInfo)) {
create(instanceManager.addHostCreatedInstance(camera2CameraInfo), reply);
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
// Copyright 2013 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

package io.flutter.plugins.camerax;

import android.content.Context;
import android.hardware.camera2.CameraCharacteristics;
import androidx.annotation.NonNull;
import androidx.annotation.OptIn;
import androidx.annotation.VisibleForTesting;
import androidx.camera.camera2.interop.Camera2CameraInfo;
import androidx.camera.camera2.interop.ExperimentalCamera2Interop;
import androidx.camera.core.CameraInfo;
import io.flutter.plugin.common.BinaryMessenger;
import io.flutter.plugins.camerax.GeneratedCameraXLibrary.Camera2CameraInfoHostApi;
import java.util.Objects;

/**
* Host API implementation for {@link Camera2CameraInfo}.
*
* <p>This class handles instantiating and adding native object instances that are attached to a
* Dart instance or handle method calls on the associated native class or an instance of the class.
*/
public class Camera2CameraInfoHostApiImpl implements Camera2CameraInfoHostApi {
private final BinaryMessenger binaryMessenger;
private final InstanceManager instanceManager;
private final Camera2CameraInfoProxy proxy;

/** Proxy for methods of {@link Camera2CameraInfo}. */
@VisibleForTesting
public static class Camera2CameraInfoProxy {

@NonNull
@OptIn(markerClass = ExperimentalCamera2Interop.class)
public Camera2CameraInfo createFrom(@NonNull CameraInfo cameraInfo) {
return Camera2CameraInfo.from(cameraInfo);
}

@NonNull
@OptIn(markerClass = ExperimentalCamera2Interop.class)
public Integer getSupportedHardwareLevel(@NonNull Camera2CameraInfo camera2CameraInfo) {
return camera2CameraInfo.getCameraCharacteristic(
CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL);
}

@NonNull
@OptIn(markerClass = ExperimentalCamera2Interop.class)
public String getCameraId(@NonNull Camera2CameraInfo camera2CameraInfo) {
return camera2CameraInfo.getCameraId();
}
}

/**
* Constructs an {@link Camera2CameraInfoHostApiImpl}.
*
* @param binaryMessenger used to communicate with Dart over asynchronous messages
* @param instanceManager maintains instances stored to communicate with attached Dart objects
* @param context {@link Context} used to retrieve {@code Executor}
*/
public Camera2CameraInfoHostApiImpl(
@NonNull BinaryMessenger binaryMessenger, @NonNull InstanceManager instanceManager) {
this(binaryMessenger, instanceManager, new Camera2CameraInfoProxy());
}

/**
* Constructs an {@link Camera2CameraInfoHostApiImpl}.
*
* @param binaryMessenger used to communicate with Dart over asynchronous messages
* @param instanceManager maintains instances stored to communicate with attached Dart objects
* @param proxy proxy for methods of {@link Camera2CameraInfo}
*/
@VisibleForTesting
Camera2CameraInfoHostApiImpl(
@NonNull BinaryMessenger binaryMessenger,
@NonNull InstanceManager instanceManager,
@NonNull Camera2CameraInfoProxy proxy) {
this.instanceManager = instanceManager;
this.binaryMessenger = binaryMessenger;
this.proxy = proxy;
}

@Override
@NonNull
public Long createFrom(@NonNull Long cameraInfoIdentifier) {
final CameraInfo cameraInfo =
Objects.requireNonNull(instanceManager.getInstance(cameraInfoIdentifier));
final Camera2CameraInfo camera2CameraInfo = proxy.createFrom(cameraInfo);
final Camera2CameraInfoFlutterApiImpl flutterApi =
new Camera2CameraInfoFlutterApiImpl(binaryMessenger, instanceManager);

flutterApi.create(camera2CameraInfo, reply -> {});
return instanceManager.getIdentifierForStrongReference(camera2CameraInfo);
}

@Override
@NonNull
public Long getSupportedHardwareLevel(@NonNull Long identifier) {
return Long.valueOf(proxy.getSupportedHardwareLevel(getCamera2CameraInfoInstance(identifier)));
}

@Override
@NonNull
public String getCameraId(@NonNull Long identifier) {
return proxy.getCameraId(getCamera2CameraInfoInstance(identifier));
}

private Camera2CameraInfo getCamera2CameraInfoInstance(@NonNull Long identifier) {
return Objects.requireNonNull(instanceManager.getInstance(identifier));
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,8 @@ public void setUp(
GeneratedCameraXLibrary.MeteringPointHostApi.setup(binaryMessenger, meteringPointHostApiImpl);
GeneratedCameraXLibrary.ResolutionFilterHostApi.setup(
binaryMessenger, new ResolutionFilterHostApiImpl(instanceManager));
GeneratedCameraXLibrary.Camera2CameraInfoHostApi.setup(
binaryMessenger, new Camera2CameraInfoHostApiImpl(binaryMessenger, instanceManager));
}

@Override
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4267,4 +4267,137 @@ static void setup(
}
}
}
/** Generated interface from Pigeon that represents a handler of messages from Flutter. */
public interface Camera2CameraInfoHostApi {

@NonNull
Long createFrom(@NonNull Long cameraInfoIdentifier);

@NonNull
Long getSupportedHardwareLevel(@NonNull Long identifier);

@NonNull
String getCameraId(@NonNull Long identifier);

/** The codec used by Camera2CameraInfoHostApi. */
static @NonNull MessageCodec<Object> getCodec() {
return new StandardMessageCodec();
}
/**
* Sets up an instance of `Camera2CameraInfoHostApi` to handle messages through the
* `binaryMessenger`.
*/
static void setup(
@NonNull BinaryMessenger binaryMessenger, @Nullable Camera2CameraInfoHostApi api) {
{
BasicMessageChannel<Object> channel =
new BasicMessageChannel<>(
binaryMessenger,
"dev.flutter.pigeon.Camera2CameraInfoHostApi.createFrom",
getCodec());
if (api != null) {
channel.setMessageHandler(
(message, reply) -> {
ArrayList<Object> wrapped = new ArrayList<Object>();
ArrayList<Object> args = (ArrayList<Object>) message;
Number cameraInfoIdentifierArg = (Number) args.get(0);
try {
Long output =
api.createFrom(
(cameraInfoIdentifierArg == null)
? null
: cameraInfoIdentifierArg.longValue());
wrapped.add(0, output);
} catch (Throwable exception) {
ArrayList<Object> wrappedError = wrapError(exception);
wrapped = wrappedError;
}
reply.reply(wrapped);
});
} else {
channel.setMessageHandler(null);
}
}
{
BasicMessageChannel<Object> channel =
new BasicMessageChannel<>(
binaryMessenger,
"dev.flutter.pigeon.Camera2CameraInfoHostApi.getSupportedHardwareLevel",
getCodec());
if (api != null) {
channel.setMessageHandler(
(message, reply) -> {
ArrayList<Object> wrapped = new ArrayList<Object>();
ArrayList<Object> args = (ArrayList<Object>) message;
Number identifierArg = (Number) args.get(0);
try {
Long output =
api.getSupportedHardwareLevel(
(identifierArg == null) ? null : identifierArg.longValue());
wrapped.add(0, output);
} catch (Throwable exception) {
ArrayList<Object> wrappedError = wrapError(exception);
wrapped = wrappedError;
}
reply.reply(wrapped);
});
} else {
channel.setMessageHandler(null);
}
}
{
BasicMessageChannel<Object> channel =
new BasicMessageChannel<>(
binaryMessenger,
"dev.flutter.pigeon.Camera2CameraInfoHostApi.getCameraId",
getCodec());
if (api != null) {
channel.setMessageHandler(
(message, reply) -> {
ArrayList<Object> wrapped = new ArrayList<Object>();
ArrayList<Object> args = (ArrayList<Object>) message;
Number identifierArg = (Number) args.get(0);
try {
String output =
api.getCameraId((identifierArg == null) ? null : identifierArg.longValue());
wrapped.add(0, output);
} catch (Throwable exception) {
ArrayList<Object> wrappedError = wrapError(exception);
wrapped = wrappedError;
}
reply.reply(wrapped);
});
} else {
channel.setMessageHandler(null);
}
}
}
}
/** Generated class from Pigeon that represents Flutter messages that can be called from Java. */
public static class Camera2CameraInfoFlutterApi {
private final @NonNull BinaryMessenger binaryMessenger;

public Camera2CameraInfoFlutterApi(@NonNull BinaryMessenger argBinaryMessenger) {
this.binaryMessenger = argBinaryMessenger;
}

/** Public interface for sending reply. */
@SuppressWarnings("UnknownNullness")
public interface Reply<T> {
void reply(T reply);
}
/** The codec used by Camera2CameraInfoFlutterApi. */
static @NonNull MessageCodec<Object> getCodec() {
return new StandardMessageCodec();
}

public void create(@NonNull Long identifierArg, @NonNull Reply<Void> callback) {
BasicMessageChannel<Object> channel =
new BasicMessageChannel<>(
binaryMessenger, "dev.flutter.pigeon.Camera2CameraInfoFlutterApi.create", getCodec());
channel.send(
new ArrayList<Object>(Collections.singletonList(identifierArg)),
channelReply -> callback.reply(null));
}
}
}

0 comments on commit 79d733c

Please sign in to comment.