-
Notifications
You must be signed in to change notification settings - Fork 110
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
8f291ba
commit 9b94bc7
Showing
1 changed file
with
33 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,33 @@ | ||
# ONNX Runtime GenAI Model Validation Example | ||
|
||
## Setup | ||
|
||
Clone this repository and navigate to the `tools/python/model_validation folder`. | ||
|
||
```bash | ||
git clone https://github.com/microsoft/onnxruntime-genai.git | ||
cd tools/python/model_validation | ||
``` | ||
|
||
In the model_validation folder, you should find the validation_tool.py script, validation_config.json file, and this README.md. | ||
|
||
### Current Support | ||
* Gemma | ||
* Llama | ||
* Mistral | ||
* Phi | ||
* Qwen | ||
|
||
### Usage - Build the Model | ||
This step creates optimized and quantized ONNX models that run with ONNX Runtime GenAI. | ||
|
||
1. In the validation_config.json file, enter the supported Hugging Face model name. Models can be found here. | ||
2. Include the path to the output folder, precision, and execution provider. | ||
|
||
Once the model is built, you can find it in path_to_output_folder/{model_name}. This should include the ONNX model data and tokenizer. | ||
|
||
### Run the Model Validation Script | ||
```bash | ||
python validation_tool.py -j validation_config.json | ||
``` | ||
|