why autoregressive_sampling and speculative_sampling_v2 output different tokens

[java63940]

Thank you for your contribution. I have a question to ask.

In your main.py, why are the outputs of autoregressive_sampling and speculative_sampling_v2 not exactly the same? Looking forward to your reply. Thank you!