Skip to content

[None][docs] drop deleted lm_eval_ad.py from auto_deploy README #18959

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
imitater-dou wants to merge 1 commit into
base: main
Choose a base branch
from
+1 −13
Open

[None][docs] drop deleted lm_eval_ad.py from auto_deploy README #18959

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 1 addition & 13 deletions examples/auto_deploy/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -92,18 +92,6 @@ The script supports flexible configs:

The default level is `INFO`.

### Model Evaluation with LM Evaluation Harness

lm-evaluation-harness is supported. To run the evaluation, please use the following command:

```bash
# model is defined the same as above. Other config args can also be specified in the model_args (comma separated).
# You can specify any tasks supported with lm-evaluation-harness.
cd examples/auto_deploy
python lm_eval_ad.py \
--model autodeploy --model_args model=meta-llama/Meta-Llama-3.1-8B-Instruct,world_size=2 --tasks mmlu
```

### Mixed-precision Quantization using Model Optimizer

Model Optimizer [AutoQuantize](https://nvidia.github.io/Model-Optimizer/reference/generated/modelopt.torch.quantization.model_quant.html#modelopt.torch.quantization.model_quant.auto_quantize) algorithm is a PTQ algorithm from ModelOpt which quantizes a model by searching for the best quantization format per-layer while meeting the performance constraint specified by the user. This way, `AutoQuantize` enables to trade-off model accuracy for performance.
Expand Down Expand Up @@ -339,4 +327,4 @@ the current progress in AutoDeploy and where you can help.
## Disclaimer

This project is under active development and is currently released as beta feature. The code is
subject to change, and may include backward-incompatible updates.
subject to change, and may include backward-incompatible updates.