AFLW20003dEvaluation.ipynb ERROR

[Chen94yue]

When I try to run AFLW20003dEvaluation.ipynb, I got this error in cell[23] with the checkpoint file download from google drive, my code is:

modelfile = '/home/Projects/neuralnet-tracker-traincode-master/repro300wlp4.ckpt'
net = trackertraincode.neuralnets.models.load_model(modelfile)
inputsize = net.input_resolution
net.cuda()
net.eval()

---------------------------------------------------------------------------
InvalidFileFormatError                    Traceback (most recent call last)
File ~/Projects/neuralnet-tracker-traincode-master/trackertraincode/neuralnets/models.py:384, in load_model(filename)
    [383](https://vscode-remote+ssh-002dremote-002b7b22686f73744e616d65223a224445563a4147585f6a706b5f6465766c6f70227d.vscode-resource.vscode-cdn.net/home/jd/Projects/neuralnet-tracker-traincode-master/scripts/~/Projects/neuralnet-tracker-traincode-master/trackertraincode/neuralnets/models.py:383) try:
--> [384](https://vscode-remote+ssh-002dremote-002b7b22686f73744e616d65223a224445563a4147585f6a706b5f6465766c6f70227d.vscode-resource.vscode-cdn.net/home/jd/Projects/neuralnet-tracker-traincode-master/scripts/~/Projects/neuralnet-tracker-traincode-master/trackertraincode/neuralnets/models.py:384)     return trackertraincode.neuralnets.io.load_model(filename, [NetworkWithPointHead])
    [385](https://vscode-remote+ssh-002dremote-002b7b22686f73744e616d65223a224445563a4147585f6a706b5f6465766c6f70227d.vscode-resource.vscode-cdn.net/home/jd/Projects/neuralnet-tracker-traincode-master/scripts/~/Projects/neuralnet-tracker-traincode-master/trackertraincode/neuralnets/models.py:385) except trackertraincode.neuralnets.io.InvalidFileFormatError as e:

File ~/Projects/neuralnet-tracker-traincode-master/trackertraincode/neuralnets/io.py:43, in load_model(filename, class_candidates)
     [42](https://vscode-remote+ssh-002dremote-002b7b22686f73744e616d65223a224445563a4147585f6a706b5f6465766c6f70227d.vscode-resource.vscode-cdn.net/home/jd/Projects/neuralnet-tracker-traincode-master/scripts/~/Projects/neuralnet-tracker-traincode-master/trackertraincode/neuralnets/io.py:42) if not all(x in contents for x in ['state_dict','class_name','config']):
---> [43](https://vscode-remote+ssh-002dremote-002b7b22686f73744e616d65223a224445563a4147585f6a706b5f6465766c6f70227d.vscode-resource.vscode-cdn.net/home/jd/Projects/neuralnet-tracker-traincode-master/scripts/~/Projects/neuralnet-tracker-traincode-master/trackertraincode/neuralnets/io.py:43)     raise InvalidFileFormatError(f'Bad dict contents. Got {list(contents.keys())}')
     [44](https://vscode-remote+ssh-002dremote-002b7b22686f73744e616d65223a224445563a4147585f6a706b5f6465766c6f70227d.vscode-resource.vscode-cdn.net/home/jd/Projects/neuralnet-tracker-traincode-master/scripts/~/Projects/neuralnet-tracker-traincode-master/trackertraincode/neuralnets/io.py:44) class_name = contents['class_name']

InvalidFileFormatError: Bad dict contents. Got ['convnet.conv1.weight', 'convnet.bn1.weight', 'convnet.bn1.bias', 'convnet.bn1.running_mean', 'convnet.bn1.running_var', 'convnet.bn1.num_batches_tracked', 'convnet.dw2_1.conv_dw.weight', 'convnet.dw2_1.bn_dw.weight', 'convnet.dw2_1.bn_dw.bias', 'convnet.dw2_1.bn_dw.running_mean', 'convnet.dw2_1.bn_dw.running_var', 'convnet.dw2_1.bn_dw.num_batches_tracked', 'convnet.dw2_1.conv_sep.weight', 'convnet.dw2_1.bn_sep.weight', 'convnet.dw2_1.bn_sep.bias', 'convnet.dw2_1.bn_sep.running_mean', 'convnet.dw2_1.bn_sep.running_var', 'convnet.dw2_1.bn_sep.num_batches_tracked', 'convnet.dw2_2.conv_dw.weight', 'convnet.dw2_2.bn_dw.weight', 'convnet.dw2_2.bn_dw.bias', 'convnet.dw2_2.bn_dw.running_mean', 'convnet.dw2_2.bn_dw.running_var', 'convnet.dw2_2.bn_dw.num_batches_tracked', 'convnet.dw2_2.conv_sep.weight', 'convnet.dw2_2.bn_sep.weight', 'convnet.dw2_2.bn_sep.bias', 'convnet.dw2_2.bn_sep.running_mean', 'convnet.dw2_2.bn_sep.running_var', 'convnet.dw2_2.bn_sep.num_batches_tracked', 'convnet.dw3_1.conv_dw.weight', 'convnet.dw3_1.bn_dw.weight', 'convnet.dw3_1.bn_dw.bias', 'convnet.dw3_1.bn_dw.running_mean', 'convnet.dw3_1.bn_dw.running_var', 'convnet.dw3_1.bn_dw.num_batches_tracked', 'convnet.dw3_1.conv_sep.weight', 'convnet.dw3_1.bn_sep.weight', 'convnet.dw3_1.bn_sep.bias', 'convnet.dw3_1.bn_sep.running_mean', 'convnet.dw3_1.bn_sep.running_var', 'convnet.dw3_1.bn_sep.num_batches_tracked', 'convnet.dw3_2.conv_dw.weight', 'convnet.dw3_2.bn_dw.weight', 'convnet.dw3_2.bn_dw.bias', 'convnet.dw3_2.bn_dw.running_mean', 'convnet.dw3_2.bn_dw.running_var', 'convnet.dw3_2.bn_dw.num_batches_tracked', 'convnet.dw3_2.conv_sep.weight', 'convnet.dw3_2.bn_sep.weight', 'convnet.dw3_2.bn_sep.bias', 'convnet.dw3_2.bn_sep.running_mean', 'convnet.dw3_2.bn_sep.running_var', 'convnet.dw3_2.bn_sep.num_batches_tracked', 'convnet.dw4_1.conv_dw.weight', 'convnet.dw4_1.bn_dw.weight', 'convnet.dw4_1.bn_dw.bias', 'convnet.dw4_1.bn_dw.running_mean', 'convnet.dw4_1.bn_dw.running_var', 'convnet.dw4_1.bn_dw.num_batches_tracked', 'convnet.dw4_1.conv_sep.weight', 'convnet.dw4_1.bn_sep.weight', 'convnet.dw4_1.bn_sep.bias', 'convnet.dw4_1.bn_sep.running_mean', 'convnet.dw4_1.bn_sep.running_var', 'convnet.dw4_1.bn_sep.num_batches_tracked', 'convnet.dw4_2.conv_dw.weight', 'convnet.dw4_2.bn_dw.weight', 'convnet.dw4_2.bn_dw.bias', 'convnet.dw4_2.bn_dw.running_mean', 'convnet.dw4_2.bn_dw.running_var', 'convnet.dw4_2.bn_dw.num_batches_tracked', 'convnet.dw4_2.conv_sep.weight', 'convnet.dw4_2.bn_sep.weight', 'convnet.dw4_2.bn_sep.bias', 'convnet.dw4_2.bn_sep.running_mean', 'convnet.dw4_2.bn_sep.running_var', 'convnet.dw4_2.bn_sep.num_batches_tracked', 'convnet.dw5_1.conv_dw.weight', 'convnet.dw5_1.bn_dw.weight', 'convnet.dw5_1.bn_dw.bias', 'convnet.dw5_1.bn_dw.running_mean', 'convnet.dw5_1.bn_dw.running_var', 'convnet.dw5_1.bn_dw.num_batches_tracked', 'convnet.dw5_1.conv_sep.weight', 'convnet.dw5_1.bn_sep.weight', 'convnet.dw5_1.bn_sep.bias', 'convnet.dw5_1.bn_sep.running_mean', 'convnet.dw5_1.bn_sep.running_var', 'convnet.dw5_1.bn_sep.num_batches_tracked', 'convnet.dw5_2.conv_dw.weight', 'convnet.dw5_2.bn_dw.weight', 'convnet.dw5_2.bn_dw.bias', 'convnet.dw5_2.bn_dw.running_mean', 'convnet.dw5_2.bn_dw.running_var', 'convnet.dw5_2.bn_dw.num_batches_tracked', 'convnet.dw5_2.conv_sep.weight', 'convnet.dw5_2.bn_sep.weight', 'convnet.dw5_2.bn_sep.bias', 'convnet.dw5_2.bn_sep.running_mean', 'convnet.dw5_2.bn_sep.running_var', 'convnet.dw5_2.bn_sep.num_batches_tracked', 'convnet.dw5_3.conv_dw.weight', 'convnet.dw5_3.bn_dw.weight', 'convnet.dw5_3.bn_dw.bias', 'convnet.dw5_3.bn_dw.running_mean', 'convnet.dw5_3.bn_dw.running_var', 'convnet.dw5_3.bn_dw.num_batches_tracked', 'convnet.dw5_3.conv_sep.weight', 'convnet.dw5_3.bn_sep.weight', 'convnet.dw5_3.bn_sep.bias', 'convnet.dw5_3.bn_sep.running_mean', 'convnet.dw5_3.bn_sep.running_var', 'convnet.dw5_3.bn_sep.num_batches_tracked', 'convnet.dw5_4.conv_dw.weight', 'convnet.dw5_4.bn_dw.weight', 'convnet.dw5_4.bn_dw.bias', 'convnet.dw5_4.bn_dw.running_mean', 'convnet.dw5_4.bn_dw.running_var', 'convnet.dw5_4.bn_dw.num_batches_tracked', 'convnet.dw5_4.conv_sep.weight', 'convnet.dw5_4.bn_sep.weight', 'convnet.dw5_4.bn_sep.bias', 'convnet.dw5_4.bn_sep.running_mean', 'convnet.dw5_4.bn_sep.running_var', 'convnet.dw5_4.bn_sep.num_batches_tracked', 'convnet.dw5_5.conv_dw.weight', 'convnet.dw5_5.bn_dw.weight', 'convnet.dw5_5.bn_dw.bias', 'convnet.dw5_5.bn_dw.running_mean', 'convnet.dw5_5.bn_dw.running_var', 'convnet.dw5_5.bn_dw.num_batches_tracked', 'convnet.dw5_5.conv_sep.weight', 'convnet.dw5_5.bn_sep.weight', 'convnet.dw5_5.bn_sep.bias', 'convnet.dw5_5.bn_sep.running_mean', 'convnet.dw5_5.bn_sep.running_var', 'convnet.dw5_5.bn_sep.num_batches_tracked', 'convnet.dw5_6.conv_dw.weight', 'convnet.dw5_6.bn_dw.weight', 'convnet.dw5_6.bn_dw.bias', 'convnet.dw5_6.bn_dw.running_mean', 'convnet.dw5_6.bn_dw.running_var', 'convnet.dw5_6.bn_dw.num_batches_tracked', 'convnet.dw5_6.conv_sep.weight', 'convnet.dw5_6.bn_sep.weight', 'convnet.dw5_6.bn_sep.bias', 'convnet.dw5_6.bn_sep.running_mean', 'convnet.dw5_6.bn_sep.running_var', 'convnet.dw5_6.bn_sep.num_batches_tracked', 'convnet.dw6.conv_dw.weight', 'convnet.dw6.bn_dw.weight', 'convnet.dw6.bn_dw.bias', 'convnet.dw6.bn_dw.running_mean', 'convnet.dw6.bn_dw.running_var', 'convnet.dw6.bn_dw.num_batches_tracked', 'convnet.dw6.conv_sep.weight', 'convnet.dw6.bn_sep.weight', 'convnet.dw6.bn_sep.bias', 'convnet.dw6.bn_sep.running_mean', 'convnet.dw6.bn_sep.running_var', 'convnet.dw6.bn_sep.num_batches_tracked', 'boxnet.linear.weight', 'boxnet.linear.bias', 'boxnet.scales.hidden_scale', 'posnet.linear_xy.weight', 'posnet.linear_xy.bias', 'posnet.linear_size.weight', 'posnet.linear_size.bias', 'posnet.scales.lin.weight', 'posnet.scales.bn.weight', 'posnet.scales.bn.bias', 'posnet.scales.bn.running_mean', 'posnet.scales.bn.running_var', 'posnet.scales.bn.num_batches_tracked', 'quatnet.linear.weight', 'quatnet.linear.bias', 'quatnet.uncertainty_net.lin.weight', 'quatnet.uncertainty_net.bn.weight', 'quatnet.uncertainty_net.bn.bias', 'quatnet.uncertainty_net.bn.running_mean', 'quatnet.uncertainty_net.bn.running_var', 'quatnet.uncertainty_net.bn.num_batches_tracked', 'landmarks.deformablekeypoints.keypts', 'landmarks.deformablekeypoints.keyeigvecs', 'landmarks.shapenet.weight', 'landmarks.shapenet.bias', 'landmarks.point_distrib_scales.hidden_scale', 'landmarks.shape_distrib_scales.hidden_scale']

During handling of the above exception, another exception occurred:

RuntimeError                              Traceback (most recent call last)
Cell In[23], [line 3](vscode-notebook-cell:?execution_count=23&line=3)
      [1](vscode-notebook-cell:?execution_count=23&line=1) # modelfile = '../model_files/pub_synface_oroi/run2/swa_NetworkWithPointHead_mobilenetv1.ckpt'
      [2](vscode-notebook-cell:?execution_count=23&line=2) modelfile = '/home/jd/Projects/neuralnet-tracker-traincode-master/repro300wlp4.ckpt'
----> [3](vscode-notebook-cell:?execution_count=23&line=3) net = trackertraincode.neuralnets.models.load_model(modelfile)
      [4](vscode-notebook-cell:?execution_count=23&line=4) inputsize = net.input_resolution
      [5](vscode-notebook-cell:?execution_count=23&line=5) net.cuda()

File ~/Projects/neuralnet-tracker-traincode-master/trackertraincode/neuralnets/models.py:387, in load_model(filename)
...
	size mismatch for posnet.linear_size.weight: copying a param with shape torch.Size([1, 1024]) from checkpoint, the shape in current model is torch.Size([1, 512]).
	size mismatch for quatnet.linear.weight: copying a param with shape torch.Size([4, 1024]) from checkpoint, the shape in current model is torch.Size([4, 512]).
	size mismatch for landmarks.shapenet.weight: copying a param with shape torch.Size([50, 1024]) from checkpoint, the shape in current model is torch.Size([50, 512]).
	size mismatch for landmarks.point_distrib_scales.hidden_scale: copying a param with shape torch.Size([68]) from checkpoint, the shape in current model is torch.Size([69]).
	size mismatch for landmarks.shape_distrib_scales.hidden_scale: copying a param with shape torch.Size([50]) from checkpoint, the shape in current model is torch.Size([51]).

[DaWelter]

I think I know ... are you on the branch https://github.com/opentrack/neuralnet-tracker-traincode/tree/paper?

Back then the code was like this:

modelfile = 'filename.ckpt'
net = trackertraincode.neuralnets.models.NetworkWithPointHead(
    enable_point_head=True,
    config='mobilenetv1', 
    enable_uncertainty=True)
net.load_state_dict(torch.load(modelfile))
inputsize = net.input_resolution
net.cuda()
net.eval()

It needs changing of the model settings to match the checkpoint.

Since then I also changed the model itself, that's why it doesn't load.

So, please try https://github.com/opentrack/neuralnet-tracker-traincode/blob/paper/scripts/AFLW20003dEvaluation.ipynb