Hi, thank you for your great work.
Today, I want to do an ablation experience on your work. I just modified the momentum_growth funtion.
from
y, idx = torch.sort(torch.abs(grad).flatten(), descending=True)
to
y, idx = torch.sort(torch.abs(grad).flatten(), descending=False)
I take the experience with the command:
python main.py --growth momentum --prune magnitude --redistribution momentum --prune-rate 0.2 --density 0.1 --data cifar10 --model vgg-c
I foud that the final sparsity will drop to 0.073. I read the source code and find that momentum_growth funtion can't growth enough weight because it didn't tell weather the mask was 0 befor growth. You deal this problem with the adjusted_growth. And I wonde that why this method work in your origin function but can't work in my ablantion experience.
Hi, thank you for your great work.
Today, I want to do an ablation experience on your work. I just modified the
momentum_growthfuntion.from
y, idx = torch.sort(torch.abs(grad).flatten(), descending=True)to
y, idx = torch.sort(torch.abs(grad).flatten(), descending=False)I take the experience with the command:
python main.py --growth momentum --prune magnitude --redistribution momentum --prune-rate 0.2 --density 0.1 --data cifar10 --model vgg-cI foud that the final sparsity will drop to 0.073. I read the source code and find that
momentum_growthfuntion can't growth enough weight because it didn't tell weather the mask was 0 befor growth. You deal this problem with theadjusted_growth. And I wonde that why this method work in your origin function but can't work in my ablantion experience.