The alternative would be to do just one stage of training with say 70:30 original/backtranslated ratio. I'll run more experiments but I think if we back-translate news with a strong student model we get a very clean back-translated corpus and still imperfect noisy original corpus. So it's understandable that the model might not beat the best checkpoint from the pre-training stage.

However, I recently found some issues with data for en-ru (for example this) so it might be another reason. cc @gregtatum