Blue Tint over entire remote video when using ARKit

[alanscarpa]

Description

I've followed the ARKit example and my app is attempting to display remote AR video.
It is successful - except for one issue: The remote video is appearing with a blue tint.

I've tried different pixel formats:

let format = VideoFormat()
format.pixelFormat = PixelFormat.format32BGRA

but the only other format that worked was format32AGRA and that actually made the blue tint an even darker blue.

One difference is that I am using an ARView to get my snapshots rather than a ARSCNView because I am using RealityKit. I don't think this would cause the issue, but I figure I show how I'm getting my snapshot:

@objc func displayLinkDidFire(timer: CADisplayLink) {
        guard let arView = self.arView else { return }
        guard arView.frame.height > 0 else { return }
        // Our capturer polls the ARSCNView's snapshot for processed AR video content, and then copies the result into a CVPixelBuffer.
        // This process is not ideal, but it is the most straightforward way to capture the output of SceneKit.
        arView.snapshot(saveToHDR: false) { image in
            guard let image = image else {
                print("No snapshot!")
                return
            }
            guard let imageRef = image.cgImage else {
                print("No cgImage from snapshot!")
                return
            }
            // A VideoSource must deliver CVPixelBuffers (and not CGImages) to a VideoSink.
            if let pixelBuffer = self.copyPixelbufferFromCGImageProvider(image: imageRef) {
                self.frame = VideoFrame(timeInterval: timer.timestamp,
                                        buffer: pixelBuffer,
                                        orientation: VideoOrientation.up)
                guard let sink = self.sink else {
                    print("No Sink!")
                    return
                }
                sink.onVideoFrame(self.frame!)
            }
        }
    }

Here's more code that is relevant:

func copyPixelbufferFromCGImageProvider(image: CGImage) -> CVPixelBuffer? {
            let dataProvider: CGDataProvider? = image.dataProvider
            let data: CFData? = dataProvider?.data
            let baseAddress = CFDataGetBytePtr(data!)

            /**
             * We own the copied CFData which will back the CVPixelBuffer, thus the data's lifetime is bound to the buffer.
             * We will use a CVPixelBufferReleaseBytesCallback callback in order to release the CFData when the buffer dies.
             **/
            let unmanagedData = Unmanaged<CFData>.passRetained(data!)
            var pixelBuffer: CVPixelBuffer? = nil
            let status = CVPixelBufferCreateWithBytes(nil,
                                                      image.width,
                                                      image.height,
                                                      PixelFormat.format32BGRA.rawValue,
                                                      UnsafeMutableRawPointer( mutating: baseAddress!),
                                                      image.bytesPerRow,
                                                      { releaseContext, baseAddress in
                                                        let contextData = Unmanaged<CFData>.fromOpaque(releaseContext!)
                                                        contextData.release() },
                                                      unmanagedData.toOpaque(),
                                                      nil,
                                                      &pixelBuffer)

            if (status != kCVReturnSuccess) {
                return nil;
            }

            return pixelBuffer
        }

Expected Behavior

I expect the remote video to not have a blue tint over it when it appears.

Actual Behavior

The video feed has a blue tint. Image of what it looks like:

Reproduces How Often

Every time.

Versions

All relevant version information for the issue.

Video iOS SDK

Twilio Video 5.0.0 via Swift Package Manager

Xcode

Xcode 14 and 13

iOS Version

iOS 15.6

iOS Device

iPhone XS and others.

Please let me know if there is any other info you'd like provided that might be helpful.

[hermiteer]

This has suddenly started happening to our app as well (iOS 17, iPhone 15 Pro, TwilioVideo 5.9.0). We aren't using anything like ARKit, just a CADisplayLink and UIGraphicsGetImageFromCurrentImageContext(). Wondering if there has been an underlying change in the pixel format of the UIView snapshot...