Skip to content

Pay-per-token Inference #30

Description

@mattjhawken

Currently, models can be requested directly via API for free on a time basis. A per-token monitoring system should be implemented, either tied to user/worker stats stored in the DHT or through some other dedicated monitor.

Validators currently maintain a list of popular Hugging Face models that have been used on the network, and these models are automatically loaded and available for free use. These free api jobs could be the target of the token monitoring and payment system.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    help wantedExtra attention is needednodePeer-to-peer node features and optimizations.torchPyTorch neural network workflow and optimizations.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions