Device error on TokenClassificationPipeline

[ierezell]

Environment info

• transformers version: 4.11.0
• Platform: Linux-5.14.8-arch1-1-x86_64-with-arch
• Python version: 3.7.11
• PyTorch version (GPU?): 1.9.1+cu102 (True)
• Tensorflow version (GPU?): not installed (NA)
• Flax version (CPU?/GPU?/TPU?): not installed (NA)
• Jax version: not installed
• JaxLib version: not installed
• Using GPU in script?: True
• Using distributed or parallel set-up in script?: False

Who can help

Library:

• pipelines: @LysandreJik

Information

Model I am using (Bert, XLNet ...):

The problem arises when using:

• the official example scripts: (give details below)

The tasks I am working on is:

• an official GLUE/SQUaD task: (give the name)

To reproduce

Steps to reproduce the behavior:

1. Create a pipe = TokenClassificationPipeline(model=DistilBertForTokenClassification.from_pretrained("PATH"))
2. Pipe some text in pipe(["My", "text", "tokens"])
3. Get a TypeError: can't convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.

Expected behavior

Be able to run the pipeline

The pipeline should bring data to gpu/cpu or model to gpu/cpu and vice versa.

The traceback

In .venv/lib/python3.7/site-packages/transformers/pipelines/token_classification.py:209 in _forward                                                                                
    206 │   │   if self.framework == "tf":                                                         
    207 │   │   │   outputs = self.model(model_inputs.data)[0][0].numpy()                          
    208 │   │   else:                                                                              
 ❱ 209 │   │   │   outputs = self.model(**model_inputs)[0][0].numpy()   <== HERE
    210 │   │   return {                                                                           
    211 │   │   │   "outputs": outputs,                                                            
    212 │   │   │   "special_tokens_mask": special_tokens_mask,                                    

Placing a .cpu() would solve the problem

Thanks in advance for any help
Have a wonderful day

[LysandreJik]

Nice catch! Would you like to open a PR with the fix?

[ierezell]

Yes, I can do it for only 6 characters

[ierezell]

Done, See pull request above: #13819

I let the CI/CD tests run as there is no new features and I didn't want to run them locally burning my pc down :)
I made it fast but tell me if anything is not okay.

Have a great day

[mallorbc]

similar issue later in the file, line 223

 220 │   │   sentence = model_outputs["sentence"]                           │
│   221 │   │   input_ids = model_outputs["input_ids"][0]                      │
│   222 │   │   offset_mapping = model_outputs["offset_mapping"][0] if model_o │
│ ❱ 223 │   │   special_tokens_mask = model_outputs["special_tokens_mask"][0].numpy() │
│   224 │   │                                                                  │
│   225 │   │   scores = np.exp(outputs) / np.exp(outputs).sum(-1, keepdims=Tr │
│   226 │   │   pre_entities = self.gather_pre_entities(                       

[ierezell]

Thanks, I committed new changes.

@LysandreJik Do you want me to also add a test (all currents tests are passing) ?

in tests/test_pipelines_token_classification.py like :

@require_torch_gpu
@slow
def test_correct_devices(self):
    sentence = "This dummy sentence checks if all the variables can be loaded on gpu and bring back to cpu"
    ner = TokenClassificationPipeline(model="distilbert-base-cased", device=0)    

[LysandreJik]

I believe this was fixed by #13856, which also implemented tests.