Could I use the same tokenizer when I finetune a RNNTBPEModel for ASR?

[bfss]

I updated the model's tokenizer during fine-tuning, but since my dataset contains only a limited vocabulary, the model's performance on the test set worsened. Should I revert to using the same tokenizer as the base model? If so, how can I accomplish this? Alternatively, is there a way to incorporate my dataset's words into the tokenizer?

[titu1994]

Skip calling change_vocabulary(). Then model will use original tokenizer

[bfss]

Thanks, I will have a try.