Add support for TensorRT v10 (multiple api calls have changed)

[remz1337]

These changes allow Frigate to use TensorRT 10.x.x models. A few changes going from TensorRT v8 to v10:

• binding_is_input has been removed
• get_binding_shape --> get_tensor_shape
• get_binding_dtype --> get_tensor_dtype

And I've added a version check to keep current behavior working for those using TensorRT v7 or v8

[netlify]

✅ Deploy Preview for frigate-docs canceled.

Name	Link
🔨 Latest commit	95ebe0c
🔍 Latest deploy log	https://app.netlify.com/sites/frigate-docs/deploys/664909f56337f20008243b5f

[NickM-27]

CC @NateMeyer can you take a look at this?

[NateMeyer]

I don't believe I can test TensorRT 10+ since they removed support for my Pascal card. Are we adding this because there will be a TensorRT v10 build of Frigate or will it just be an alternate build/fork that swaps in v10?

[remz1337]

When installing Frigate manually, TensorRT might have already been installed with the latest version (v10), hence why I would like to include support. I would like to avoid patching this file everytime I update Frigate.

[blakeblackshear]

To confirm, what do you mean by "installing Frigate manually"?

[remz1337]

To confirm, what do you mean by "installing Frigate manually"?

We've developed a script at Tteck's Proxmox Helper Script to install Frigate without Docker. For now, it works with CPU and Coral detectors, and I'm working on adding support for TensorRT. See discussion at tteck/Proxmox#2711

[blakeblackshear]

In general, we wouldn't accept code changes for unsupported install methods. We have rejected PRs in the past for similar reasons. Since this is a community supported detector, if @NateMeyer is ok with the added maintenance burden, then I think it's ok.

[NateMeyer]

I'm not opposed to having the support for v10 in here, there will come a day we need to upgrade TensorRT to support newer cards. I'm a little unsure of how best to handle it. The current PR is reasonable, but I wonder if we could reduce the duplicate code and refactor it a bit to have the shape, dtype, and input checks happen in separate small functions.

[remz1337]

My suggestion to reduce code duplication would be to check TRT_VERSION before every tensorRT api call that has changed (binding_is_input, get_binding_shape and get_binding_dtype). This would increase the number of TRT_VERSION checks but reduce code duplication.

I'll work on it later this week

[NateMeyer]

My suggestion to reduce code duplication would be to check TRT_VERSION before every tensorRT api call that has changed (binding_is_input, get_binding_shape and get_binding_dtype). This would increase the number of TRT_VERSION checks but reduce code duplication.

I'll work on it later this week

I was thinking have three new local functions, such as get_shape(), get_dtype(), check_input() or something, that would do the TRT version check inside.

[remz1337]

Updated

[NateMeyer]

This looks good to me.