MediaMetadata: Support using a XML drawable as artwork

[y20k]

Use case description

I develop maintain an audio app that lets users add their own media. Not all user provided media come with an artwork. In that case my app uses a default vector drawable to represent the media in the UI. The app's default drawable makes use of dynamic colors to better fit in with the theme. The default icon should not only be used in the app's user interface, but also in the notification and in other clients, that connect to the playback service. I think there is a whole class of apps that need to provide a default fallback icon for media artwork.

Proposed solution

setArtworkUri in MediaMetadata.Builder could accept Uris that point to android XML resources.

Alternatives considered

The app could grab the XML default drawable, create a Bitmap, convert it to a ByteArray and feed that to setArtworkData to build the MediaMetadata for media that has no artwork.

[marcbaechinger]

Thanks for your request!

I wouldn't recommend using artworkData if it can be avoided. The reason for this is that the MediaItem is transferred from the session to the controller and including a byte array bloats the media item and can make this transfer slow primarily in cases with many media items in the playlist. This would be specifically unnecessary when the byte array contains the same default bitmap many times.

Short answer/quickest and available solution: Store that bitmap image on an http server on the internet and reference it by URI.

Long answer/(probably) best solution:

Better would IMO be to only use artworkUri and then let controllers decide whether/how to download and cache the bytes.

Personally I think the best and most efficient solution would be to store the bitmap in your app and create a content provider and serve the image with the content provider. UAMP does this for album artwork. The the artwortUri can be set to a content URI like content://best.radio.app/path/to/my/artwork/1234. This is IMO the high-quality approach we should aim for and that the library should make as easy and efficient as possible. That's efficient because no bytes are contained in the media item (even not in MediaMeatadataCompat!). That's neat because well supported by Android UI components, APIs and clients (SystemUI, Auto, WearOS), allows sharing across processes and works offline.

From this I think the enhancement for the Media3 library would probably be to make sure that content:// URI are supported properly and (by accident :P) we are pretty close to that I think. We recently added DataSourceBitmapLoader that is able to download from content:// URIs. I don't think someone ever has used it yet with MediaSession.Builder.setBitmapLoader though. :) I would guess it should work, even when the library probably not yet takes full advantage of it (see 2 below).

Without thinking more about it than since starting writing this comment, I think there are these AIs around this enhancement for us (or the rest of the world that wants that):

1 - Modify the session demo app or UAMP to use DataSourceBitmapLoader
2 - Make sure that MediaSessionLegacyImpl and DefaultMediaNotificationProvider do not download artworkUris with content:// scheme URIs in artworkUri but instead just leave this to clients.
3 - Verify that SystemUI (media notification), Android Auto, Wear OS and other clients are happy with content:// URIs (I think that works as per UAMP). Minimal API buckets to test notification with: API 21-29/29-32/33+
4 - Remove workarounds in UAMP and only use content:// URIs

If you depend on the source of the main branch you can use the DataSourceBitmapLoader already with MediaSession.Builder.setBitmapLoader. So if you are eager to test this... :)

[y20k]

Hi @marcbaechinger

Thank you for your answer!!! 🙂

best and most efficient solution would be to store the bitmap in your app and create a content provider and serve the image with the content provider.

I think that already works for PNG images. I tried that a couple of days ago. But it does not work for vector drawables. Why not use PNGs? With vector drawables the fallback default icon can adapt to the app's theme. In my case a themed fallback icon is already used in the phone UI. It would be great if this icon could be used in the notification, too.

I know that this might be out of scope for media3, but I does not hurt to ask.

In any case it is great to read that you consider to officially support content:// Uris in artworkUri!

[marcbaechinger]

Ah, I see! Sorry I misunderstood. :D Yeah, that would require a platform change. As a library we'd still need to support the API levels before this new feature with bitmaps.

[y20k]

Just a quick addition:

setArtworkUri(Uri) handles given local Uris in my experience (Android 13) as follows:

Type	Result
android.resource://	given artwork is displayed(*)
file://	given artwork is displayed(*) / MediaDataManager throws FileNotFoundException (Permission denied) is thrown
content:// (FileProvider)	given artwork is not displayed / logcat does not show an error

(*) actually sometimes the artwork embedded in the actual source is displayed and the artwork set via setArtworkUri(Uri) is ignored. I have not yet found out, when that happens.

[y20k]

@marcbaechinger

Yeah, that would require a platform change. As a library we'd still need to support the API levels before this new feature with bitmaps.

Yes, that makes sense. I guess I can close this feature request then.

While working with setArtworkUri I stumbled upon another issue: setArtworkUri does not really seem to support local files at all. Since this is a different issue I created a new feature request: #271