Cannot pickle TFAutomodels

[Killua7362]

from transformers import TFAutoModel, AutoTokenizer   
checkpoint_name = "hf-internal-testing/tiny-random-MobileBertModel"
model = TFAutoModel.from_pretrained(checkpoint_name)

Above is the example code, it works with AutoModel from transformers. I cannot pickle TFAutoModel
Below is the error
Layer 'embeddings' expected 0 variables, but received 3 variables during loading. Expected: []

[mmckerns]

I think this is an issue for transformers you may want to report it to huggingface with dill.detect.trace(True).