feat: Create Frame.withBaseClass to ensure it's always a FrameHostObject

[mrousavy]

What

Adds the Frame.withBaseClass(..) method, which takes a jsi::Object and uses the given jsi::Object as it's new base class.

Before, there were two approaches for combining objects in JS:

1. Combine with spread:

   const frame = ...
   const canvas = surface.getCanvas()
   const drawableFrame = { ...frame, ...canvas } 

   This is eager, and slow as it has to iterate through all C++ properties and methods of both frame and canvas and copies them here, even if you don't use them.
2. Use Proxy:

   const frame = ...
   const canvas = surface.getCanvas()
   const drawableFrame = new Proxy({}, {
     get: (property, target) => {
       return frame[property] ?? canvas[property]
     }
   })

   Unfortunately Proxies are also slow, but with well implemented Proxies this lazy-approach should be better and more flexible than the spread approach.

Unfortunately both of these approaches have a common problem; the drawableFrame value is no longer a FrameHostObject/jsi::HostObject, but instead a conventional jsi::Object now. It will also no longer hold the native shared_ptr of buffer data.

This causes two issues:

• The drawableFrame cannot be passed to native Frame Processor Plugins, because those expect a FrameHostObject, not a normal jsi::Object. That should've thrown an error actually, maybe in some runtimes it was swallowed? Not sure
• Using the Frame outside of the useFrameProcessor context (e.g. runAsync) wouldn't work anymore, as the drawableFrame is not a HostObject that can be easily shared. Instead, it was a conventional jsi::Object that had to be deep-copied.

So instead, we need to make sure that the frame stays the instance it currently is, and we do that by adding a new method to FrameHostObject: withBaseClass.

This basically just uses the object it receives and sets it as a class property, and all properties that do not exist on FrameHostObject will now be looked up on the new jsi::Object base class.

This means that the code is now:

const frame = ...
const canvas = surface.getCanvas()
const drawableFrame = frame.withBaseClass(canvas)

..and drawableFrame is now still a FrameHostObject - so it can be passed to Frame Processor Plugins without an issue. Also runAsync should now work fine.

Changes

Tested on

Related issues

• Fixes 🐛 Function inside runAsync in useSkiaFrameProcessor runs only once #2842

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
react-native-vision-camera	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	May 6, 2024 5:07pm