Question about Padding Penalty

[violet-sto]

Hi,

Thanks for the excellent work! I notice that you use 'padding penalty' to control the length of the generated sequences, and I want to try it. However, I didn't find the corresponding code. If I miss it, please point out the location.

Thanks

[zhxieml]

Hi @violet-sto, to enable padding penalty during inference, please set eos_penalty > 0 as outlined here: https://huggingface.co/Dream-org/Dream-Coder-v0-Instruct-7B/blob/main/generation_utils.py#L422.

[violet-sto]

Thanks for the reply! I also want to know how to set the hyperparameter eos_penalty. As shown in the paper (Figure1b), the penalty is set to 1e-12. I want to know whether it is too small.

[zhxieml]

I want to know whether it is too small.

We found [1e-12, 1e-3] to be a reasonable range.