Added NetInt Quadra Support #1539

n8o · 2024-02-27T18:52:22Z

This PR adds basic support for NetInt Quadra.

Added Docker Image
Updated docs
Tested AVC ABR encoding with 8bit and 10bit content.
Decode, Scale, Encode all in hardware

Needs:

Tests. Where are the unit tests?
Test HEVC encoding and decoding
Add JPEG and AV1 Support

getroot · 2024-02-29T11:50:22Z

Thanks for your contribution. Unfortunately we don't have this device so we can't test it. We have no choice but to rely on contributors for this equipment.

We don't have a unit test template yet, and each developer is doing his own testing.

AV1 cannot be tested because no publisher supports it yet. We will support this for WebRTC in the future.

@nums Could you please review this?

nums · 2024-03-01T14:17:51Z

Yes, I will do a review.

I was also not able to test directly: unfortunately my netint card is a "logan" but not a "quadra"

nums · 2024-03-01T14:23:10Z

hello @n8o , I did a quick check and I noticed this line:
https://github.com/n8o/OvenMediaEngine/blob/master/src/projects/transcoder/codec/encoder/encoder_avc_niquadra.cpp#L78
On the avc side it is commented but you kept it on the hevc.
On the Logan, I had to apply it on both.

It's probably normal but in case it's forgotten after a test...

n8o · 2024-03-01T15:01:37Z

What line are you referring to? L7 is a copyright header line. I haven't actually tested HEVC decode and encode. I'll do that today and should be done before merging.

nums · 2024-03-01T15:07:12Z

What line are you referring to? L7 is a copyright header line. I haven't actually tested HEVC decode and encode. I'll do that today and should be done before merging.

sorry, bad copy and paste: https://github.com/n8o/OvenMediaEngine/blob/master/src/projects/transcoder/codec/encoder/encoder_avc_niquadra.cpp#L78

n8o · 2024-03-01T15:14:06Z

I should be able to enable that line. I don't think it matters. I believe i disabled it while debugging issues before i got it working. the low latency setting is probably nice for webrtc. not sure if it affects quality or not for ABR HLS.

I believe OME already forces keyframes when they are needed. so the keyframe param may not be needed.
https://github.com/n8o/OvenMediaEngine/blob/59e7368a7494c6b1d7b2c434b70b5715a97fae32/src/projects/transcoder/codec/encoder/encoder_avc_niquadra.cpp#L40

nums · 2024-03-01T15:25:17Z

For the lowdelay parameter I didn't see any difference but I added it just in case, however the gopPresetIdx allowed b-frames to be deactivated. (for the record, it seems to me that the gopPreset is at 4)

n8o · 2024-03-01T16:54:57Z

I tried enabling it and got a bunch of errors. Looking at the documentation for quadra it reports:

lowDelay
Specifies whether or not to enable the low latency mode in encoding. When enabled, libxcoder increases its rate of polling the encoder and only permits buffering of a single frame to minimize the delay.

NOTE – That when enabled, the gopPresetIdx must be 1, 3, 7, 9, 10, or 0 with a consecutive order gop pattern, lookaheadDepth must be 0, and multicoreJointMode must be 0.

Supported values for gopPresetIdx are:

-1: Adaptive Gop (default)
0 : Custom Gop
1 : I-I-I-I,..I (all intra, gop_size=1)
3 : I-B-B-B,...B (consecutive B, gop_size=1)
4 : I-B-P-B-P,... (gop_size=2)
5 : I-B-B-B-P,... (gop_size=4)
7 : I-B-B-B-B,... (consecutive B, gop_size=4)
8 : I-B-B-B-B-B-B-B-B,... (random access, gop_size=8) 9 : I-P-P-P,... P (consecutive P, gop_size=1)
10 : I-P-P-P-P,... (hierarchical P, gop_size=4)

So with that, I set it to 1 and 1:
::av_opt_set(_codec_context->priv_data, "xcoder-params", "gopPresetIdx=1:lowDelay=1", 0);

that seems to work.

n8o · 2024-03-05T19:17:28Z

Any more changes or requests?

n8o · 2024-03-07T15:17:29Z

@nums or @getroot

bump

nums · 2024-03-07T15:30:52Z

Sorry, a little bit busy this week.

@getroot, on my side it's ok and ready to be merge :)

n8o · 2024-03-07T15:33:47Z

Actually i just noticed that gopPresetIdx 1 is all i-frames. I need to change that. Either idx 9 or 10?

nums · 2024-03-07T15:38:44Z

yes, 10 seems to be the same than 2 (https://github.com/AirenSoft/OvenMediaEngine/blob/master/src/projects/transcoder/codec/encoder/encoder_avc_nilogan.cpp#L48C2-L48C17) on logan so better than 1

…ays to use 9 with lowDelay

n8o · 2024-03-07T17:25:12Z

Fixed. Documentation said to use 9 with lowDelay.

basisbit

Just a few comments.

basisbit · 2024-03-07T23:52:32Z

Dockerfile.dev

@@ -0,0 +1,45 @@
+# Instruction: Build local OvenMediaEngine Dev Environment. Can be used with CLion dev environment as a toolchain
+# Information on hot to setup: https://www.jetbrains.com/help/clion/clion-toolchains-in-docker.html#sample-dockerfile


hot to should probably be how to

basisbit · 2024-03-07T23:54:37Z

Dockerfile.dev

@@ -0,0 +1,45 @@
+# Instruction: Build local OvenMediaEngine Dev Environment. Can be used with CLion dev environment as a toolchain


To be honest, I am not too fond of having yet another Dcokerfile here which will have to be manually updated each time to keep in sync with the other Dockerfiles and the docker-compose.yml. There already exists the Dockerfile.local for local building and developing.

That makes sense. I will merge them and maybe add a comment on how to only do a build image vs release image.
https://docs.docker.com/build/building/multi-stage/

basisbit · 2024-03-07T23:58:01Z

Dockerfile.niquadra.local

@@ -0,0 +1,111 @@
+# Instruction: Build OvenMediaEngine with Netint Quadra support


Instead of having many Dockerfiles, please consider moving anything hardware acceleration specific which is not an environment parameter into one of the misc/prerequisites scripts to keep it easy to get OME running in those environments which are either not using Docker or are not one of the "standard" supported environments.

I'll investigate this

basisbit · 2024-03-08T00:03:46Z

Dockerfile.niquadra.local

+
+#yasm install
+WORKDIR /NI_Release
+RUN wget -c http://www.tortall.net/projects/yasm/releases/yasm-1.3.0.tar.gz


please change this to https, there is no hash / checksum checking here.

basisbit · 2024-03-08T00:06:52Z

misc/prerequisites.sh

+            cd $NETINT_QUADRA_XCODER_COMPILE_PATH && bash build.sh && ldconfig #the compilation of libxcoder can be done before
+        fi
+        ADDI_LIBS+=" --enable-ni_quadra --enable-avfilter  --enable-pthreads "
+        ADDI_ENCODER+=",h264_ni_quadra,h265_ni_quadra"


What are the license implications of this? Is releasing a pre-built docker container with this compliant wit AGPL?

I'm not sure about licensing and how this works for AGPL. wouldn't this be the same as all the others? Nvidia driver is proprietary. so is the logan and quadra. OME uses it through ffmpeg which is compiled with nvidia and netint support. i do believe the one header file that ffmpeg uses for nvidia is open source. but everything else is close source. I'm not affiliated with netint. so i'm not sure

I just assumed since you already had logan support that quadra would be fine. this section is basically just a copy of how logan's prerequisites work..

basisbit · 2024-03-08T00:09:42Z

src/projects/transcoder/filter/filter_rescaler.cpp

+        switch (input_module_id)
+        {
+            case cmn::MediaCodecModuleId::NIQUADRA: { 	// Zero Copy
+                //TODO: Exception handling required if Device ID is different.


Please finish this before merging, thanks

basisbit · 2024-03-08T00:11:57Z

src/projects/transcoder/transcoder_stream.cpp


-				output_track->SetColorspace(encoder->GetSupportedFormat());
-			}
+            }


was this indentation change on purpose? If not, please revert

basisbit · 2024-03-08T00:14:34Z

Also, might be worth it to update this PR to ffmpeg 6 or newer

getroot · 2024-03-08T13:35:02Z

Also, might be worth it to update this PR to ffmpeg 6 or newer

ffmpeg 6 will take some time for us.

stale · 2024-05-08T03:26:44Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

n8o and others added 6 commits February 20, 2024 08:31

Adding NetInt Quadra to the build scripts

5af2a60

First attempt at quadra

d935488

Finished adding netint quadra functionality

2126912

Merge branch 'AirenSoft:master' into master

9ce797f

Added documentation

e9740d8

Removed debug log

84f4005

n8o and others added 2 commits February 29, 2024 07:20

Merge branch 'AirenSoft:master' into master

183cc62

Added documentation

59e7368

Added xcoder params back. Fixed Quadra HEVC encoding.

29ed61b

Fixed gopPresetIdx to 9. 1 was creating all i-frames. Documentation s…

6356f0c

…ays to use 9 with lowDelay

basisbit suggested changes Mar 8, 2024

View reviewed changes

stale bot added the stale label May 8, 2024

getroot removed the stale label May 13, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added NetInt Quadra Support #1539

Added NetInt Quadra Support #1539

n8o commented Feb 27, 2024

getroot commented Feb 29, 2024

nums commented Mar 1, 2024

nums commented Mar 1, 2024 •

edited

n8o commented Mar 1, 2024

nums commented Mar 1, 2024

n8o commented Mar 1, 2024

nums commented Mar 1, 2024

n8o commented Mar 1, 2024

n8o commented Mar 5, 2024

n8o commented Mar 7, 2024

nums commented Mar 7, 2024

n8o commented Mar 7, 2024

nums commented Mar 7, 2024

n8o commented Mar 7, 2024

basisbit left a comment

basisbit Mar 7, 2024

basisbit Mar 7, 2024

n8o Mar 8, 2024

basisbit Mar 7, 2024

n8o Mar 8, 2024

basisbit Mar 8, 2024

basisbit Mar 8, 2024

n8o Mar 8, 2024

basisbit Mar 8, 2024

basisbit Mar 8, 2024

basisbit commented Mar 8, 2024

getroot commented Mar 8, 2024

stale bot commented May 8, 2024

		@@ -0,0 +1,45 @@
		# Instruction: Build local OvenMediaEngine Dev Environment. Can be used with CLion dev environment as a toolchain
		# Information on hot to setup: https://www.jetbrains.com/help/clion/clion-toolchains-in-docker.html#sample-dockerfile

		@@ -0,0 +1,111 @@
		# Instruction: Build OvenMediaEngine with Netint Quadra support

Added NetInt Quadra Support #1539

Are you sure you want to change the base?

Added NetInt Quadra Support #1539

Conversation

n8o commented Feb 27, 2024

getroot commented Feb 29, 2024

nums commented Mar 1, 2024

nums commented Mar 1, 2024 • edited

n8o commented Mar 1, 2024

nums commented Mar 1, 2024

n8o commented Mar 1, 2024

nums commented Mar 1, 2024

n8o commented Mar 1, 2024

n8o commented Mar 5, 2024

n8o commented Mar 7, 2024

nums commented Mar 7, 2024

n8o commented Mar 7, 2024

nums commented Mar 7, 2024

n8o commented Mar 7, 2024

basisbit left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

basisbit commented Mar 8, 2024

getroot commented Mar 8, 2024

stale bot commented May 8, 2024

nums commented Mar 1, 2024 •

edited