You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I use accelerate for accelerated calculations, using a single card. In accelerate, I use deepspeed settings to use bf16 precision for lora fine-tuning and inference. However, when I save the model and reload it for inference, the speed is very slow from 7min to 40min , and the results are different from the original results, and the results cannot be reproduced. The same random seed is set in training and loading model inference.
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.
Please note that issues that do not follow the contributing guidelines are likely to be ignored.
deepspeed 0.13.0
accelerate 0.24.1
peft 0.7.1
I use
accelerate
for accelerated calculations, using a single card. In accelerate, I usedeepspeed
settings to use bf16 precision forlora
fine-tuning and inference. However, when I save the model and reload it for inference, the speed is very slow from 7min to 40min , and the results are different from the original results, and the results cannot be reproduced. The same random seed is set in training and loading model inference.lora:
save model:
load model:
deepspeed config:
The text was updated successfully, but these errors were encountered: