[expo-media-library] [ios] show if an asset is an icloud asset

[jarvisluong]

So I've been building an app which need to create a media picker. The problem is when user's phone has images or videos stored on icloud (and not yet locally). In order to display those icloud asset properly I would need to wait until the asset is downloaded (as already defined in here:

expo/packages/expo-media-library/ios/EXMediaLibrary/EXMediaLibrary.m

Line 407 in e166a8d

options.networkAccessAllowed = YES;

To listen for download updates, it is also already possible since expo-media-library has a delegated method:

expo/packages/expo-media-library/ios/EXMediaLibrary/EXMediaLibrary.m

Line 567 in e166a8d

- (void)photoLibraryDidChange:(PHChange *)changeInstance

My suggestion would be to add this key isIcloudAsset: boolean to the return value of MediaLibrary.getAssetInfoAsync(asset), so that the user of this library know that this asset needs to wait for download before showing it. In order to do this, we can check this key (PHImageResultIsInCloudKey) inside the info object of this callback:

expo/packages/expo-media-library/ios/EXMediaLibrary/EXMediaLibrary.m

Line 422 in e166a8d

[[PHImageManager defaultManager] requestAVAssetForVideo:asset

I'm happy to include a PR to about this issue as well.

Reference: https://www.swiftjectivec.com/icloud-photo-handling/

[tsapeta]

Hey! Yeah, it sounds like a good idea! 🙂
However I wouldn't use iCloud in this key — who knows, maybe one day we will be able to add similar thing on Android? This way we will avoid any breaking changes 😉 What about isLocalAsset, isNetworkAsset, needsDownloading?

[jarvisluong]

That's a valid point, I would say isNetworkAsset fits in this case

[tsapeta]

Great, feel free to open a PR if you'd like!

[awinograd]

Something like isNetworkAsset would solve my use case in #8769 as well. However, I wonder if there's a suggested "workaround" for taking thumbnails / previewing the file without having to download it in its entirety. Like @jarvisluong I'm also creating a custom media picker and really just want to render a thumbnail of the remote asset and download it only if it's selected by the user.

[jarvisluong]

Interesting point, I will be working on this on the weekend so will look into that

[byCedric]

Thanks @jarvisluong! Let me know if I can help 😄

[jarvisluong]

@awaisabir @byCedric The ability to generate a thumbnail is certainly possible since there is an API for that: https://developer.apple.com/documentation/photokit/phimagemanager/1616964-requestimageforasset?language=objc

Also, I see that this is already implemented in expo-image-manipulator:

expo/packages/expo-image-manipulator/ios/EXImageManipulator/EXImageManipulatorModule.m

Line 73 in 8fa0b00

[[PHImageManager defaultManager] requestImageForAsset:asset targetSize:size contentMode:PHImageContentModeAspectFit options:options resultHandler:^(UIImage * _Nullable image, NSDictionary * _Nullable info) {

So basically for now we can use ImageManipulator.manipulateAsync() with empty actions, and we will have the thumbnail for the iCloud assets. I don't think there is a need to reimplement the same functionality in expo-media-library (though certainly some documentation could be helpful as well). The need to add isNetworkAsset is still certainly required and I will make a PR for this.

[jarvisluong]

One more thing, also thanks to this PR: #8304, if we just want the thumbnail for the asset, upgrading to latest expo-media-library will allow you to pass the uri in Asset type to Image component and you can get the thumbnail easily

[awinograd]

@jarvisluong thanks for your investigation and information!

I was able to find pretty good success generating thumbnails for icloud videos with expo-image-picker@8.1.0. However, sometimes (~20%) i'd still get a blank thumbnail (without an error being thrown). For what it's worth I did find a slightly better "success" rate when resizing the image to my exact needs vs maintaining the original size.

I updated to expo-image-picker@8.2.1 and saw an even better "success" rate. Maybe 90% instead of 80%?

I'm not really familiar with native iOS development or the APIs available so sorry I'm not adding more intelligent feedback / commentary to the discussion, but:

1. if [media-library] Add missing image loader for MediaLibrary in bare workflow #8304 is what added support for generating properly accessing network assets in ImagePicker I'm not sure why I saw success with 8.1.0
2. I'm not sure why i'm still seeing an occasional blank thumbnail when using ImageManipulator.manipulateAsync on 8.2.1. I wonder if some network error is being swallowed and it's not bubbling up as an error to the JS promise?

Edit:

RE (2), I tried running manipulateAsync on an interval and noted that blank thumbnails were replaced successfully with proper thumbnails after the first retry and that a properly generated thumbnail was never replaced with a blank thumbnail by a subsequent retry. Makes me my theory about the failed network request may be correct and that once the asset is fetched successfully, the thumbnail will not fail to generate

[jarvisluong]

@awinograd Sorry if my comments are making the informations misleading, but actually what I mean was:

• ImageManipulator.manipulateAsync used the exact native API which is used for generate thumbnail of an Asset (which is PHAsset in iOS domain). This is not related to the versioning of expo-media-library

• expo-image-library@8.2 added a new asset loader which allows React native's <Image /> component to consume the asset.uri as source directly (without the manual thumbnail generation from the developer side). Again, the loader of expo-image-library also use the same native API from the one which is already implemented in expo-image-manipulator

For your point (1), since the call getAssetInfoAsync triggered a network download in iOS native side already, so later call may already have the downloaded version of the assets.

I suggest you can try to get the thumbnail like this: (with version 8.2.1 of expo-media-library)

<Image source={{uri: asset.uri}} /> // notice the .uri, not localUri

[awinograd]

@jarvisluong I must have misread your comments incorrectly. Thanks for the extra clarification. Using <Image source={{ asset.uri}} /> directly seems to be the winner!

Thanks again for your help on this issue.