Skip to content

[FEATURE] Improve setup_models.py to build engines #108

Description

@eliteprox

Feature type

Build

Describe the feature

setup_models.py should both download and build engines

Motivation

Currently there is a loose dependency between ai-runner and comfystream with models.yaml and dl_checkpoints.sh. A change to models.yaml can break dl_checkpoints and the ai-runner CI.

dl_checkpoints already reuses setup_models.py to download the models https://github.com/livepeer/ai-runner/blob/9e21e45480bf3112388f5b4b6bfb11df3ea51522/runner/dl_checkpoints.sh#L101-L111, however it doesn't compile them.

dl_checkpoints separately compiles tensorrt engines and assumes the model files are present. https://github.com/livepeer/ai-runner/blob/9e21e45480bf3112388f5b4b6bfb11df3ea51522/runner/dl_checkpoints.sh#L136-L145

models.yaml can be updated to include the necessary parameters to compile the engines so that build_trt is only ran for models which were also downloaded by the config.

Additional context

No response

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions