QGC doesn't display video stream

[adolgarev]

Hello,

I followed instructions in the README and tried t3.12.7 and t3.12.6 releases but QGC doesn't automatically display the video stream. I create ball-Fake source h264 rtp stream and can add it successfully manually in QGC settings, QGC then displays the stream as expected. But automatically this doesn't happen. I see that mavlink-camera-manager sends heartbeats and QGC reacts to them and requests MAV_CMD_REQUEST_CAMERA_INFORMATION. In mavlink-camera-manager logs

2024-03-10T04:22:38.616028Z TRACE           MavReceiver ThreadId(13) receiver_loop: src/mavlink/manager.rs:104: Message received: MavHeader { system_id: 255, component_id: 190, sequence: 11 }, COMMAND_LONG(COMMAND_LONG_DATA { param1: 1.0, param2: 0.0, param3: 0.0, param4: 0.0, param5: 0.0, param6: 0.0, param7: 0.0, command: MAV_CMD_REQUEST_CAMERA_INFORMATION, target_system: 1, target_component: 100, confirmation: 0 })

But mavlink-camera-manager neither acknowledges the message not sends the information back. I can verify this by watching messages on mavproxy to which mavlink-camera-manager is connected. As a result QGC reports "Giving up requesting camera info from 1 100"

Do you know what can be the problem? Thanks.

See the full log mavlink-camera-manager.2024-03-10-04.log attached

[joaoantoniocardoso]

Hi! Thanks for reporting.

I create ball-Fake source h264 rtp stream and can add it successfully

You mean, rtsp? I just submitted a PR fixing it.

[joaoantoniocardoso]

@adolgarev could you try the new version (t3.12.8)? Then we reopen if it is still not working.

Thanks!

[joaoantoniocardoso]

@adolgarev I released a new version (t3.12.9) because the previous fix wasn't enough for older GStreamer versions (like 1.23.4).

[adolgarev]

@joaoantoniocardoso thank you for your response. Recent build produces the same result, please find details below.

I configure the stream like below

I create ball-Fake source h264 rtp stream and can add it successfully
You mean, rtsp? I just submitted a PR fixing it.

I believe it is RTP over UDP

If I add stream manually

It displays ok

But if I let QGC discover stream by itself I see

and no video

In logs mavlink-camera-manager sends only heartbeats, no message ack and no response with camera caps.

System id and component id looks right. I wonder why mavlink-camera-manager doesn't respond.

The new log:
mavlink-camera-manager.2024-03-11-19.log

[joaoantoniocardoso]

Hi, I'm able to reproduce the issue, I'll investigate it.

[joaoantoniocardoso]

@adolgarev can you test #357 ?

• mavlink-camera-manager-aarch64

• mavlink-camera-manager-armv7

• mavlink-camera-manager-linux-desktop

[adolgarev]

It works! Thank yo very much.

There is one warning in QGC which is not fatal but maybe worth a look.

[joaoantoniocardoso]

@adolgarev from my tests, some unresponsiveness on the mavlink camera manager side makes the messages from the QGC be interpreted and answered too late, thus failing to start the stream.

We are still working on that.

[joaoantoniocardoso]

@adolgarev from my tests, some unresponsiveness on the mavlink camera manager side makes the messages from the QGC be interpreted and answered too late, thus failing to start the stream.

We are still working on that.

this PR solves this.

@adolgarev, the Unable to load camera parameters from camera definition is something that I saw during the development of this fix, but I'm not seeing it anymore, so I believe it was related to the performance issue.

[adolgarev]

Thanks! I'll give it a try

[adolgarev]

Can you please make a build with the PR above? Thanks

[joaoantoniocardoso]

@adolgarev Now the changes are in master:

mavlink-camera-manager-aarch64
mavlink-camera-manager-armv7
mavlink-camera-manager-linux-desktop

[adolgarev]

I still see CameraControlLog: Unable to load camera parameters from camera definition on QGC, I believe it is because I'm using fake source.

[patrickelectric]

@adolgarev but the video is showing ? Or only the controls ?

[DmitriyLugovoy]

hello!
I modified src/stream/pipeline/v4l_pipeline.rs::76
VideoEncodeType::Yuyv => { format!( concat!( "v4l2src device={device} do-timestamp=true", " ! videoconvert", " ! x264enc", " ! h264parse", " ! capsfilter name={filter_name} caps=video/x-h264,stream-format=avc,alignment=au,width={width},height={height},framerate={interval_denominator}/{interval_numerator}", " ! tee name={video_tee_name} allow-not-linked=true", " ! rtph264pay aggregate-mode=zero-latency config-interval=10 pt=96", " ! tee name={rtp_tee_name} allow-not-linked=true" ),
and src/stream/sink/image_sink.rs::355
VideoEncodeType::Yuyv => { // For h264, we need to filter-out unwanted non-key frames here, before decoding it. let filter = gst::ElementFactory::make("identity") .property("drop-buffer-flags", gst::BufferFlags::DELTA_UNIT) .property("sync", false) .build()?; let decoder = gst::ElementFactory::make("avdec_h264") .property_from_str("lowres", "2") // (0) is 'full'; (1) is '1/2-size'; (2) is '1/4-size' .build()?; decoder.has_property("discard-corrupted-frames", None).then(|| decoder.set_property("discard-corrupted-frames", true)); _transcoding_elements.push(filter); _transcoding_elements.push(decoder); }
Now I'm able to stream from YUYV to QGC.

Also I updated src/mavlink/mavlink_camera.rs::279
flags: mavlink::common::CameraCapFlags::CAMERA_CAP_FLAGS_HAS_VIDEO_STREAM |
mavlink::common::CameraCapFlags::CAMERA_CAP_FLAGS_CAN_CAPTURE_IMAGE_IN_VIDEO_MODE |
mavlink::common::CameraCapFlags::CAMERA_CAP_FLAGS_HAS_BASIC_ZOOM |
mavlink::common::CameraCapFlags::CAMERA_CAP_FLAGS_HAS_BASIC_FOCUS,

what I need to do to see zoom and focus controls in QGC?
Will appreciate for your help!

[DmitriyLugovoy]

here logs of QGC and camera_def.xml and camera-manager logs
QGC_logs.txt
HP HD Camera: HP HD Camera_HP HD Camera: HP HD Camera_000.zip
mavlink-camera-manager.2024-03-22-09.log

[joaoantoniocardoso]

hello! I modified src/stream/pipeline/v4l_pipeline.rs::76 VideoEncodeType::Yuyv => { format!( concat!( "v4l2src device={device} do-timestamp=true", " ! videoconvert", " ! x264enc", " ! h264parse", " ! capsfilter name={filter_name} caps=video/x-h264,stream-format=avc,alignment=au,width={width},height={height},framerate={interval_denominator}/{interval_numerator}", " ! tee name={video_tee_name} allow-not-linked=true", " ! rtph264pay aggregate-mode=zero-latency config-interval=10 pt=96", " ! tee name={rtp_tee_name} allow-not-linked=true" ), and src/stream/sink/image_sink.rs::355 VideoEncodeType::Yuyv => { // For h264, we need to filter-out unwanted non-key frames here, before decoding it. let filter = gst::ElementFactory::make("identity") .property("drop-buffer-flags", gst::BufferFlags::DELTA_UNIT) .property("sync", false) .build()?; let decoder = gst::ElementFactory::make("avdec_h264") .property_from_str("lowres", "2") // (0) is 'full'; (1) is '1/2-size'; (2) is '1/4-size' .build()?; decoder.has_property("discard-corrupted-frames", None).then(|| decoder.set_property("discard-corrupted-frames", true)); _transcoding_elements.push(filter); _transcoding_elements.push(decoder); } Now I'm able to stream from YUYV to QGC.

Also I updated src/mavlink/mavlink_camera.rs::279 flags: mavlink::common::CameraCapFlags::CAMERA_CAP_FLAGS_HAS_VIDEO_STREAM | mavlink::common::CameraCapFlags::CAMERA_CAP_FLAGS_CAN_CAPTURE_IMAGE_IN_VIDEO_MODE | mavlink::common::CameraCapFlags::CAMERA_CAP_FLAGS_HAS_BASIC_ZOOM | mavlink::common::CameraCapFlags::CAMERA_CAP_FLAGS_HAS_BASIC_FOCUS,

what I need to do to see zoom and focus controls in QGC? Will appreciate for your help!

Cool, I'll check your logs soon, but can you make a draft PR with your changes?

The zoom should show up if it is shown on the front-end page hosted by MCM.. as in, it will be shown if the camera has the zoom as a UCV/V4L2 control. (it also is possible that we are doing something wrong and not detecting that control for that camera somehow..)

[DmitriyLugovoy]

ERROR: Permission to mavlink/mavlink-camera-manager.git denied to DmitriyLugovoy.

[joaoantoniocardoso]

ERROR: Permission to mavlink/mavlink-camera-manager.git denied to DmitriyLugovoy.

You need to fork the project, sorry =)

[adolgarev]

@adolgarev but the video is showing ? Or only the controls ?

Yes, video is showing

[joaoantoniocardoso]

@adolgarev but the video is showing ? Or only the controls ?

Yes, video is showing

@adolgarev good, so I won't bother looking into that. Feel free to create new issues if you find anything else, or if you want to discuss new features, thanks!

[DmitriyLugovoy]

@joaoantoniocardoso https://github.com/mavlink/mavlink-camera-manager/pull/369/files