Saving tract_onnx model #1094
-
|
As I understand it, when reading the onnx model, an unnecessary part for training is discarded from it and only what is needed for inference remains. Is it possible to save such a model to disk? If so, will this model take a size smaller? |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
|
You can dump a model in nnef form, right after decluttering, before optimising(). The size shoud be roughly the same, most of a model size is made of the trained parameters. On the other hand, you may get a shorter loading time because the expensive decluttering is already done... You can look for inspiration here: https://github.com/sonos/tract/tree/main/examples/nnef-dump-mobilenet-v2 . This show how to load NNEF model. For converting and dumping your model, load it as usual, but call https://docs.rs/tract-core/latest/tract_core/model/typed/type.TypedModel.html#method.into_decluttered instead of into_optimized() (optimized model are NOT serializable to NNEF). Then you can use https://docs.rs/tract-nnef/latest/tract_nnef/framework/struct.Nnef.html#method.write_to_tar . Alternatively, you can "cook" your model to nnef with tract command line. |
Beta Was this translation helpful? Give feedback.
You can dump a model in nnef form, right after decluttering, before optimising(). The size shoud be roughly the same, most of a model size is made of the trained parameters.
On the other hand, you may get a shorter loading time because the expensive decluttering is already done...
You can look for inspiration here: https://github.com/sonos/tract/tree/main/examples/nnef-dump-mobilenet-v2 . This show how to load NNEF model.
For converting and dumping your model, load it as usual, but call https://docs.rs/tract-core/latest/tract_core/model/typed/type.TypedModel.html#method.into_decluttered instead of into_optimized() (optimized model are NOT serializable to NNEF). Then you can use https://do…