Skip to content

rollout进程的oom bug #56

Description

@AragornHorse

我用两卡80GB跑qwen2.5-7b的rollout,长度开到了4096,也不是很长,单独运行gen_worker进程连20000token都是没问题的,用现在代码的mp.Process启动就会rollout时遇到oom,这是什么原因引起的?
我已经确认了训练时rollout进程和train进程没有共用任何显卡。

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions