Deformable DETR is now available in HuggingFace Transformers #160
Comments
|
cool ! |
|
Hi, would it be possible to obtain a detailed script to train from a pre-trained model? as following the tutorial used in DETR causes problems. |
Can you clarify which issues you had? |
|
I have changed DetrForObjectDetection to DeformableDetrForObjectDetection in the DETR class and I have also changed the DetrFeatureExtractor to AutoFeatureExtractor. Finally I define the batch size as 1 in the dataloaders and the training runs correctly but when evaluating I get the following error: IndexError: max(): Expected reduction dim 2 to have non-zero size. Looking at the inference script I see that the AutoFeatureExtractor is used to get the input and pass it to the model, because of this I am not very clear. I attach the notebook I am using so that you can see it easier. |
|
Is there any script to draw attention map of deformable detr in powerful HuggingFace? |
You can just follow this notebook where I show how to visualize attention maps of the decoder. Make sure to replace |
I think the way to draw attention map of deformble detr is much different from detr. Since it uses reference points and sampling offsets. |
|
Hi, I'm new to hugging face so I might be missing something obvious but when I try to import Also the feature extractor doesn't work as Solved it by installing with |
|
Hi, Thanks for reporting. We indeed fixed Deformable DETR's feature extractor as seen in #19140. It will be included in the next PyPi release. |
|
Alright, good to hear! By the way kudos for the great work. |
|
@NielsRogge |
|
Hi, Hmm normally that shouldn't be changed because of the feature extractor. Did you create a regular PyTorch dataset? |
|
Yes, It's basically just a number of images and their annotations. I have tested the annotations file via multiple coco-viewers and also the same thing did not happen while fine-tuning DETR with the same dataset. |
Were you using the transformers library? I am trying to export to onnx but it results in an error. Could you please share your script if you can? |
|
have you tried to export the model into torchscript format to a C++ environment for inference? i tried,but the exported model doesn't work,so l'd like to find some help from you! |
|
Could it be that you traced the model with fixed batch size of 237 and you are trying to run inference with a batch of size 32? @Zalways |
thank u for your reply! it seems not this reason cause the error, i tried input a tensor in specific shape,it still error. |
are the other images the same shape as the one you used for tracing? |
|
@andrearosasco yes , when i use other image,image has been preprocessed as the same, and i tried to input the same shape tensor into the model,it doesn't work! it's very strange, now i have no idea about it .i hope someone could help me with this issue |
|
when i try to export the deformable detr model into torchscript,it shows the error message! anybody knows how to solve it? Traceback of TorchScript, original code (most recent call last): so i think the problem maybe occurs in export step :Could not export Python function call 'MSDeformAttnFunction' looking forward to your reply! |
Hi,
Deformable DETR is now available in 馃 Transformers: https://huggingface.co/docs/transformers/main/en/model_doc/deformable_detr.
All checkpoints are on the hub: https://huggingface.co/models?other=deformable_detr.
The implementation supports both CPU and GPU (and you can choose to use the custom kernel or not when running on GPU). 馃コ
Inference
For inference, I refer to the example code snippet in the docs.
Fine-tuning on custom data
For fine-tuning, I refer to this demo notebook, illustrating how to fine-tune the model. Fine-tuning Deformable DETR is equivalent to fine-tuning DETR (just replace
DetrForObjectDetectionin the notebook byDeformableDetrForObjectDetection).The text was updated successfully, but these errors were encountered: