[Fix]: Send model output to cpu before numpy cast in token_clf_pipeline

[ierezell]

What does this PR do?

Fixes #13816

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Device error on TokenClassificationPipeline #13816
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings. No docs changes
• Did you write any new necessary tests? No additional test needed

Who can review?

If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of who to tag.
Please tag fewer than 3 people.

Library:

• pipelines: @LysandreJik

[ierezell]

@Narsil Do you want me to also add a test (all currents tests are passing) ?

in tests/test_pipelines_token_classification.py like :

@require_torch_gpu
@slow
def test_correct_devices(self):
    sentence = "This dummy sentence checks if all the variables can be loaded on gpu and bring back to cpu"
    ner = TokenClassificationPipeline(model="distilbert-base-cased", device=0)    

Have a great day

[Narsil]

Hi @ierezell .

Unfortunately this fix is not desirable, as we're trying to have all "CPU/GPU" logic be contained inside the Pipeline class (in src/transformers/pipelines/base.py).

I added another proposed fix (the test is still not ran as part of the unit tests (it's still slow) pinging @LysandreJik for advice on if/how we can make GPU tests part of the unit tests.

(I also added you as co-author as you figured out the issue and a solid fix)

[ierezell]

HI @Narsil, indeed it seems better to handle all the logic in the same place. I should've checked before.

I looked at your fix and it's perfect for me.

For the test, unfortunately, even if it's only a really small computation, the test needs GPU... Is it this bad to have it only as CI/CD test and not as a unit test?

Thanks for keeping me as coauthor.

Have a great day!

[Narsil]

Making ALL pipeline tests GPU as unit regular tests is something that was already raised internally.

Currently pipeline tests are relatively fast, so it really is doable. Slow tests are run on every release + once a day if I am not mistaken so it's already something.

[LysandreJik]

Closing as resolved by #13856. Let me know if you'd like to reopen this.