-
Notifications
You must be signed in to change notification settings - Fork 2.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Text2Topic : a new loss function ? #2605
Comments
Hi, correct me if I'm wrong but I think this is already implemented in the SoftmaxLoss class, you just have to set all the
Implementation aside, the paper reminds me a lot of approaches from SphereFace2, and exploiting hyperspherical embeddings for OOD, which are pretty cool. |
Oh I see. The simplest way to do this is a bit hacky. You can just use the SoftmaxLoss and add then change the
|
Thanks, I implemented the same way ! However, I found interesting the idea to compute a similarity matching towards classifieur like Cross-Encodeur but with Bi-Encodeur for cached some embeddings. Imo, the classifieur allows to catch more information that cosine similarity. Maybe a new features for SetFit, or Sentence Transformers ? |
Since it's such a small change to the SoftmaxLoss, maybe it's better to just expose cc @tomaarsen |
Paper : https://aclanthology.org/2023.emnlp-industry.10.pdf
Hi
Recently, Booking.com shared a new architecture called Text2Topic. This architecture takes as input a text and a topic with its description, and outputs a score between 0 and 1.
To achieve this, the architecture implements a new loss function, which resembles the SoftMax loss but for binary classification.
Based on my personal experiences, the loss function also improves the embeddings of a pre-trained model like "domain adaptation".
I would like to hear opinions on the relevance of their approach and whether an implementation like the loss function would be feasible.
Thanks
The text was updated successfully, but these errors were encountered: