ipsec: Fix IPsec reinitialization

[joamaki]

Fixes issues in previous PR (#25744) on IPsec ENI BPF program loading:

loader: In IPsec reload ignore veth devices & fix settle wait
    
reloadIPSecOnLinkChanges() did not ignore veth device updates causing
reload to be triggered when new endpoints were created. Ignore any
updates with "veth" as device type.

The draining of updates during settle wait was broken due to unintentional
breaking out of the loop. Removed the break.

loader: Do not fatal on IPsec reinitialization

Now that the code is reloading the bpf_network program at runtime we
should not fatal if we fail to reload the program since this may be caused
by ongoing interface changes (e.g. interface was being removed). Change
the log.Fatal into log.Error and keep loading to other interfaces.

Validated these fixes in AWS ENI environment with IPsec enabled by creating and
destroying dummy devices:

$ for i in $(seq 1 100); do sleep 0.1; ip link add dummy$i type dummy; ip link del dummy$i; done
$ kubectl get logs ...
2023-06-06T13:55:03.474507212Z level=info msg="Reinitializing IPsec due to device changes" subsys=datapath-loader
2023-06-06T13:55:03.480640541Z level=info msg="Encryption network program (re)loaded" interface=ens3 subsys=datapath-loader
2023-06-06T13:55:03.480648286Z level=error msg="Load encryption network failed" error="getting interface dummy1 by name: Link not found" interface=dummy1 subsys=datapath-loader
2023-06-06T13:55:03.480656250Z level=warning msg="Failed to reinitialize IPsec after device change" error="failed to load encryption program: getting interface dummy1 by name: Link not found" subsys=datapath-loader
2023-06-06T13:55:04.479609463Z level=info msg="Reinitializing IPsec due to device changes" subsys=datapath-loader
2023-06-06T13:55:04.482395149Z level=info msg="Encryption network program (re)loaded" interface=ens3 subsys=datapath-loader
2023-06-06T13:55:05.482543443Z level=info msg="Reinitializing IPsec due to device changes" subsys=datapath-loader

The time between reinitializations is now at least 1 second and we do not crash when loading fails unexpectedly.

Fix three issues in the bug fix to attach IPsec BPF programs to ENI interfaces: do not fatal if loading unexpectedly fails (which may happen if the device is suddenly deleted), ignore veth device changes in order not to reinitialize when new endpoints appear and wait 1 second for further device state changes between reinitializations.

[joamaki]

/test

[rgo3]

Just left non-blocking comments, as they don't address anything functional.

[pchaigno]

@dylandreimerink Please don't trust MLH blindly. I had yet to re-review here.

[pchaigno]

LGTM.