RuntimeError: Error(s) in loading state_dict for UNetArch:

[Kidbye20]

(led) root@00ee9d88811f:~/LED# python scripts/image_process.py -p checkpoints/pretrained/LED_Pretrain_None_None_CVPR20_Setting_Ratio1-200.pth --data_path images/test/ -opt options/base/network_g/unet.yaml
Building network...
Loading checkpoint...
Loading UNetArch model from checkpoints/pretrained/LED_Pretrain_None_None_CVPR20_Setting_Ratio1-200.pth, with param key: [params].
Traceback (most recent call last):
File "scripts/image_process.py", line 115, in
image_process()
File "/root/anaconda3/envs/led/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
return func(*args, **kwargs)
File "scripts/image_process.py", line 91, in image_process
load_network(network_g, args.pretrained_network, param_key='params' if not args.led else 'params_deploy')
File "scripts/image_process.py", line 37, in load_network
net.load_state_dict(load_net, strict=strict)
File "/root/anaconda3/envs/led/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1671, in load_state_dict
raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for UNetArch:
Missing key(s) in state_dict: "conv1_1.weight", "conv1_1.bias", "conv1_2.weight", "conv1_2.bias", "conv2_1.weight", "conv2_1.bias", "conv2_2.weight", "conv2_2.bias", "conv3_1.weight", "conv3_1.bias", "conv3_2.weight", "conv3_2.bias", "conv4_1.weight", "conv4_1.bias", "conv4_2.weight", "conv4_2.bias", "conv5_1.weight", "conv5_1.bias", "conv5_2.weight", "conv5_2.bias", "conv6_1.weight", "conv6_1.bias", "conv6_2.weight", "conv6_2.bias", "conv7_1.weight", "conv7_1.bias", "conv7_2.weight", "conv7_2.bias", "conv8_1.weight", "conv8_1.bias", "conv8_2.weight", "conv8_2.bias", "conv9_1.weight", "conv9_1.bias", "conv9_2.weight", "conv9_2.bias".
Unexpected key(s) in state_dict: "conv1_1.main_weight", "conv1_1.main_bias", "conv1_1.align_weights.0", "conv1_1.align_weights.1", "conv1_1.align_weights.2", "conv1_1.align_weights.3", "conv1_1.align_weights.4", "conv1_1.align_weights.5", "conv1_1.align_biases.0", "conv1_1.align_biases.1", "conv1_1.align_biases.2", "conv1_1.align_biases.3", "conv1_1.align_biases.4", "conv1_1.align_biases.5", "conv1_2.main_weight", "conv1_2.main_bias", "conv1_2.align_weights.0", "conv1_2.align_weights.1", "conv1_2.align_weights.2", "conv1_2.align_weights.3", "conv1_2.align_weights.4", "conv1_2.align_weights.5", "conv1_2.align_biases.0", "conv1_2.align_biases.1", "conv1_2.align_biases.2", "conv1_2.align_biases.3", "conv1_2.align_biases.4", "conv1_2.align_biases.5", "conv2_1.main_weight", "conv2_1.main_bias", "conv2_1.align_weights.0", "conv2_1.align_weights.1", "conv2_1.align_weights.2", "conv2_1.align_weights.3", "conv2_1.align_weights.4", "conv2_1.align_weights.5", "conv2_1.align_biases.0", "conv2_1.align_biases.1", "conv2_1.align_biases.2", "conv2_1.align_biases.3", "conv2_1.align_biases.4", "conv2_1.align_biases.5", "conv2_2.main_weight", "conv2_2.main_bias", "conv2_2.align_weights.0", "conv2_2.align_weights.1", "conv2_2.align_weights.2", "conv2_2.align_weights.3", "conv2_2.align_weights.4", "conv2_2.align_weights.5", "conv2_2.align_biases.0", "conv2_2.align_biases.1", "conv2_2.align_biases.2", "conv2_2.align_biases.3", "conv2_2.align_biases.4", "conv2_2.align_biases.5", "conv3_1.main_weight", "conv3_1.main_bias", "conv3_1.align_weights.0", "conv3_1.align_weights.1", "conv3_1.align_weights.2", "conv3_1.align_weights.3", "conv3_1.align_weights.4", "conv3_1.align_weights.5", "conv3_1.align_biases.0", "conv3_1.align_biases.1", "conv3_1.align_biases.2", "conv3_1.align_biases.3", "conv3_1.align_biases.4", "conv3_1.align_biases.5", "conv3_2.main_weight", "conv3_2.main_bias", "conv3_2.align_weights.0", "conv3_2.align_weights.1", "conv3_2.align_weights.2", "conv3_2.align_weights.3", "conv3_2.align_weights.4", "conv3_2.align_weights.5", "conv3_2.align_biases.0", "conv3_2.align_biases.1", "conv3_2.align_biases.2", "conv3_2.align_biases.3", "conv3_2.align_biases.4", "conv3_2.align_biases.5", "conv4_1.main_weight", "conv4_1.main_bias", "conv4_1.align_weights.0", "conv4_1.align_weights.1", "conv4_1.align_weights.2", "conv4_1.align_weights.3", "conv4_1.align_weights.4", "conv4_1.align_weights.5", "conv4_1.align_biases.0", "conv4_1.align_biases.1", "conv4_1.align_biases.2", "conv4_1.align_biases.3", "conv4_1.align_biases.4", "conv4_1.align_biases.5", "conv4_2.main_weight", "conv4_2.main_bias", "conv4_2.align_weights.0", "conv4_2.align_weights.1", "conv4_2.align_weights.2", "conv4_2.align_weights.3", "conv4_2.align_weights.4", "conv4_2.align_weights.5", "conv4_2.align_biases.0", "conv4_2.align_biases.1", "conv4_2.align_biases.2", "conv4_2.align_biases.3", "conv4_2.align_biases.4", "conv4_2.align_biases.5", "conv5_1.main_weight", "conv5_1.main_bias", "conv5_1.align_weights.0", "conv5_1.align_weights.1", "conv5_1.align_weights.2", "conv5_1.align_weights.3", "conv5_1.align_weights.4", "conv5_1.align_weights.5", "conv5_1.align_biases.0", "conv5_1.align_biases.1", "conv5_1.align_biases.2", "conv5_1.align_biases.3", "conv5_1.align_biases.4", "conv5_1.align_biases.5", "conv5_2.main_weight", "conv5_2.main_bias", "conv5_2.align_weights.0", "conv5_2.align_weights.1", "conv5_2.align_weights.2", "conv5_2.align_weights.3", "conv5_2.align_weights.4", "conv5_2.align_weights.5", "conv5_2.align_biases.0", "conv5_2.align_biases.1", "conv5_2.align_biases.2", "conv5_2.align_biases.3", "conv5_2.align_biases.4", "conv5_2.align_biases.5", "conv6_1.main_weight", "conv6_1.main_bias", "conv6_1.align_weights.0", "conv6_1.align_weights.1", "conv6_1.align_weights.2", "conv6_1.align_weights.3", "conv6_1.align_weights.4", "conv6_1.align_weights.5", "conv6_1.align_biases.0", "conv6_1.align_biases.1", "conv6_1.align_biases.2", "conv6_1.align_biases.3", "conv6_1.align_biases.4", "conv6_1.align_biases.5", "conv6_2.main_weight", "conv6_2.main_bias", "conv6_2.align_weights.0", "conv6_2.align_weights.1", "conv6_2.align_weights.2", "conv6_2.align_weights.3", "conv6_2.align_weights.4", "conv6_2.align_weights.5", "conv6_2.align_biases.0", "conv6_2.align_biases.1", "conv6_2.align_biases.2", "conv6_2.align_biases.3", "conv6_2.align_biases.4", "conv6_2.align_biases.5", "conv7_1.main_weight", "conv7_1.main_bias", "conv7_1.align_weights.0", "conv7_1.align_weights.1", "conv7_1.align_weights.2", "conv7_1.align_weights.3", "conv7_1.align_weights.4", "conv7_1.align_weights.5", "conv7_1.align_biases.0", "conv7_1.align_biases.1", "conv7_1.align_biases.2", "conv7_1.align_biases.3", "conv7_1.align_biases.4", "conv7_1.align_biases.5", "conv7_2.main_weight", "conv7_2.main_bias", "conv7_2.align_weights.0", "conv7_2.align_weights.1", "conv7_2.align_weights.2", "conv7_2.align_weights.3", "conv7_2.align_weights.4", "conv7_2.align_weights.5", "conv7_2.align_biases.0", "conv7_2.align_biases.1", "conv7_2.align_biases.2", "conv7_2.align_biases.3", "conv7_2.align_biases.4", "conv7_2.align_biases.5", "conv8_1.main_weight", "conv8_1.main_bias", "conv8_1.align_weights.0", "conv8_1.align_weights.1", "conv8_1.align_weights.2", "conv8_1.align_weights.3", "conv8_1.align_weights.4", "conv8_1.align_weights.5", "conv8_1.align_biases.0", "conv8_1.align_biases.1", "conv8_1.align_biases.2", "conv8_1.align_biases.3", "conv8_1.align_biases.4", "conv8_1.align_biases.5", "conv8_2.main_weight", "conv8_2.main_bias", "conv8_2.align_weights.0", "conv8_2.align_weights.1", "conv8_2.align_weights.2", "conv8_2.align_weights.3", "conv8_2.align_weights.4", "conv8_2.align_weights.5", "conv8_2.align_biases.0", "conv8_2.align_biases.1", "conv8_2.align_biases.2", "conv8_2.align_biases.3", "conv8_2.align_biases.4", "conv8_2.align_biases.5", "conv9_1.main_weight", "conv9_1.main_bias", "conv9_1.align_weights.0", "conv9_1.align_weights.1", "conv9_1.align_weights.2", "conv9_1.align_weights.3", "conv9_1.align_weights.4", "conv9_1.align_weights.5", "conv9_1.align_biases.0", "conv9_1.align_biases.1", "conv9_1.align_biases.2", "conv9_1.align_biases.3", "conv9_1.align_biases.4", "conv9_1.align_biases.5", "conv9_2.main_weight", "conv9_2.main_bias", "conv9_2.align_weights.0", "conv9_2.align_weights.1", "conv9_2.align_weights.2", "conv9_2.align_weights.3", "conv9_2.align_weights.4", "conv9_2.align_weights.5", "conv9_2.align_biases.0", "conv9_2.align_biases.1", "conv9_2.align_biases.2", "conv9_2.align_biases.3", "conv9_2.align_biases.4", "conv9_2.align_biases.5".
模型权重无法加载

[schuy1er]

Please refer to #6 which is similar to this problem.

[ioctl-user]

Run inside docker with all installation steps passed.

The same problem with models LED_Deploy_ELD_NikonD850_CVPR20_Setting_Ratio1-200.pth , LED_Deploy_SID_SonyA7S2_CVPR20_Setting_Ratio100-300.pth , LED_Deploy_SID_SonyA7S2_MM22_Setting_Ratio100-300.pth .

At that, all other Deploy models from release 0.1.0 works fine.

In all cases script image_process.py was run with only -p, --data_path and --save_path parameters. Is this correct?

[Srameo]

If you want to use the LED finetuned network, you need to add --led at the end of the parameters. We mentioned this issue in #6 , but it was discussed in Chinese.

We have mentioned this in the README.md.

LED/README.md

Line 177 in e052f27

--led if you are using a checkpoint fine-tuned by our led.

[Srameo]

The reason why it's necessary to specify --led is that the deployed model contains two checkpoints: one before structural reparameterization and one after. When running the image_process.py script, it's required to use the weights from after the reparameterization, hence the need to specify --led.

[ioctl-user]

Parameter --led is working with LED_Deploy_ELD_NikonD850_CVPR20_Setting_Ratio1-200.pth , LED_Deploy_SID_SonyA7S2_CVPR20_Setting_Ratio100-300.pth, LED_Deploy_SID_SonyA7S2_MM22_Setting_Ratio100-300.pth .

But not working with release 0.1.1 LED+NAFNet_Deploy_SID_SonyA7S2_CVPR20_Setting_Ratio100-300.pth and LED+Restormer_Deploy_SID_SonyA7S2_CVPR20_Setting_Ratio100-300.pth .

[Srameo]

To use architectures other than UNet, like Restormer/NAFNet. you need to specify the -opt options.

LED/README.md

Lines 198 to 199 in 75f1081

	-opt NETWORK_OPTIONS, --network_options NETWORK_OPTIONS
	the arch options of the pretrained network, DEFAULT: 'options/base/network_g/unet.yaml'

And we have detailed options for Restormer/NAFNet in commit 75f1081.

LED/README.md

Line 186 in 75f1081

> **Notice that**, if you wish to use the model from release v0.1.1, you need to add the `-opt` parameter: For NAFNet, add `-opt options/base/network_g/nafnet.yaml`. For Restormer, add `-opt options/base/network_g/restormer.yaml`.