[FEATURE REQ] Raw announce filtering.

[zip6como]

Is your feature request related to a problem? Please describe.
Autobrr's filtering options can't filter for a specific part of the announce that aren't tags.
So, for example, if a torrent announce starts with "Torrent featured:" autobrr can't filter this announce just by that.

Describe the solution you'd like
I would like to be able to filter the raw announce. No tags, no categories, nothing like that. Just the announce line.
For example, if a torrent announce happens with "Torrent featured:" at the beginning, I would like to be able to find it by entering *featured* as the filtering option. It would search the whole announce line and look if the string "featured" is in it.

Describe alternatives you've considered
I tried filtering by Tags and Raw Release Tags, but both couldn't find the "Torrent featured:" at the beginning.

[zze0s]

Hey! Which indexer is this for specifically? We don't have any definitions that has torrent featured in the examples, so you would not match any of those either way. Some announce channels do have other types of announces that we ignore because we/users only expect new releases.

We previously had this implemented for testing, see #807.

Quoting Kyle from that PR:

This is a huge anti-feature. Yes, it can be used in limited stop-gap situations to get people around bugs in autobrr, but exposing everything about the announce means that our definitions are not good enough for usage with the rest of the filtering. The one exception to this is RSS (and potentially Torznab) where we don't know the strings beforehand, and require user-intervention for the feeds to extract the parts out that are desired.

I might be wrong, but this feels actively harmful and encourages bad behaviour.

[zip6como]

Which indexer is this for specifically?

It's for AnimeWorld

We don't have any definitions that has torrent featured in the examples, so you would not match any of those either way.

Well it's actually "Torrent vorgestellt:" or the "[Vorgestellt]" tag (no announce has both tag and prefix, it's always only one)(neither get found by autobrr), but I translated it to English for the example.

Some announce channels do have other types of announces that we ignore because we/users only expect new releases.

So that means all announces except for new uploads get ignored?

I might be wrong, but this feels actively harmful and encourages bad behaviour.

What is meant by "actively harmful and encourages bad behaviour"?

[zze0s]

It's for AnimeWorld

I see. Do you have a full example of that line?

So that means all announces except for new uploads get ignored?

Yes, we only look for their Neuer Upload lines. We currently have no way to differentiate if it's a new or promotion etc and I'm sure users would start to wonder why an older torrent suddenly starts to download if we start to look for the featured/promoted releases.

We could probably add support for it but we'd need to plan it out.

What is meant by "actively harmful and encourages bad behaviour"?

Maybe @KyleSanderson can clarify that.

[zip6como]

I see. Do you have a full example of that line?

There are 2 types of featured torrents. Ones that get featured at upload and ones that get featured at a later point (typically a few minutes).
Torrents that get featured at upload look like this:
Neuer Upload: Spy.Classroom.S01.German.DL.DTS.1080p.BluRay.10bit.x265-CNY [Kategorie: Anime Serien][Typ: Encode][Auflösung: 1080p][Größe: 13.01 GiB][FL: 100%][Vorgestellt] | http://animeworld.cx/torrents/39924 | Uploader: Anonym
While uploads that get featured after uploading look like this (Most torrents get featured after uploading):
Torrent vorgestellt: My.Hero.Academia.S01.German.DL.DTS.1080p.BluRay.10bit.x265-FumoFriday [Kategorie: Anime Serien][Typ: Encode][Auflösung: 1080p][Größe: 21.87 GiB] | http://animeworld.cx/torrents/39884
Torrents that get featured after uploading have their upload announced first (without the [Vorgestellt] tag) and then have a second announce stating that the torrent got featured (lines that start with "Torrent vorgestellt:")

Yes, we only look for their Neuer Upload lines. We currently have no way to differentiate if it's a new or promotion etc and I'm sure users would start to wonder why an older torrent suddenly starts to download if we start to look for the featured/promoted releases.

Ah, that makes sense.

We could probably add support for it but we'd need to plan it out.

I would suggest adding a switch to include featured torrents.
But announces would still need to be filtered by type because there's also re-seed request announces, which no one wants.

[KyleSanderson]

What is meant by "actively harmful and encourages bad behaviour"?

Maybe @KyleSanderson can clarify that.

My pleasure.

This is a huge anti-feature. Yes, it can be used in limited stop-gap situations to get people around bugs in autobrr, but exposing everything about the announce means that our definitions are not good enough for usage with the rest of the filtering. The one exception to this is RSS (and potentially Torznab) where we don't know the strings beforehand, and require user-intervention for the feeds to extract the parts out that are desired.

[zip6como]

but exposing everything about the announce means that our definitions are not good enough for usage with the rest of the filtering.

But why can't there be both raw announce filtering and definition based filtering?

[KyleSanderson]

Wouldn't you like something that's well maintained by your application, than something fiddly that 90 people have to update manually when they change how this is laid out. There's now thousands of users, support requests because this has been carefully considered are very minimal all things considered now.

This situation is well supported today, it's simply appending the preamble as you've suggested to tags. If autobrr can't handle that, it's a bug.

[zip6como]

Wouldn't you like something that's well maintained by your application, than something fiddly that 90 people have to update manually when they change how this is laid out. There's now thousands of users, support requests because this has been carefully considered are very minimal all things considered now.

Ah, okay, I just thought that you could add it as an optional and experimental feature.

This situation is well supported today, it's simply appending the preamble as you've suggested to tags. If autobrr can't handle that, it's a bug.

To my understanding, "Torrent vorgestellt:" at the beginning of the line isn't a tag.

[KyleSanderson]

To my understanding, "Torrent vorgestellt:" at the beginning of the line isn't a tag.

so translate it to "featured" and throw it in the tags, the yaml should handle that today. It's an application, anything can be done. Wildcarding for Vorgestellt is just incorrect and you'll get bad matches.

[zip6como]

so translate it to "featured" and throw it in the tags, the yaml should handle that today.

I wouldn't translate it, since the tracker is meant to be in German. It would only confuse users if they had to enter *featured* instead of *vorgestellt*

It's an application, anything can be done.

So that means you could implement searching for both "Torrent vorgestellt:" and "[Vorgestellt]" and match the filter if either one is in the announce?

Wildcarding for Vorgestellt is just incorrect and you'll get bad matches.

Yeah, I understand how that could happen.

[martylukyy]

Well it's actually "Torrent vorgestellt:" or the "[Vorgestellt]" tag (no announce has both tag and prefix, it's always only one)(neither get found by autobrr), but I translated it to English for the example.

This is wrong. The [Vorgestellt] tag is getting parsed as you can see in this test.

autobrr/internal/indexer/definitions/animeworld.yaml

Lines 72 to 83 in 1530e95

	- line: 'Neuer Upload: The.Zero.S04.German.DL.DTS.1080p.BluRay.10bit.x265.Repack-ABC [Kategorie: Anime Serien][Typ: Encode][Auflösung: 1080p][Größe: 17.28 GiB][FL: 100%][Vorgestellt] | https://animeworld.cx/torrents/00000 | Uploader: Anonym'
	expect:
	torrentName: 'The.Zero.S04.German.DL.DTS.1080p.BluRay.10bit.x265.Repack-ABC'
	category: Anime Serien
	releaseTags: Encode
	resolution: 1080p
	torrentSize: 17.28 GiB
	freeleechPercent: 100
	tags: "Vorgestellt"
	baseUrl: https://animeworld.cx/
	torrentId: "00000"
	uploader: Anonym

Does that solve the issue?

[zip6como]

Oh, I didn't notice that. However, this still can't filter for lines starting with "Torrent vorgestellt:" is there any way to do that?

[martylukyy]

No. That being said, autobrr was never meant to download older releases.
I would advise you to ask staff/uploaders to not apply the featured status after the torrent has been uploaded,
but setting the featured status at the point of uploading.

[zip6como]

No. That being said, autobrr was never meant to download older releases.

So that means there is no way for me to autosnatch these releases?

I would advise you to ask staff/uploaders to not apply the featured status after the torrent has been uploaded,
But setting the featured status at the point of uploading.

Not possible since not only staff uploads get featured, but only staff can feature torrents at upload.

[martylukyy]

I currently don't see a possibility for it, no.

[zip6como]

Okay. Well then I suppose you can close the issue

[zze0s]

You could setup a custom definition and copy paste the existing indexer, set a new identifier + name and modify the irc parse lines to look for the Torrent vorgestellt: lines.

That way you could use it and filter for it explicitly, or make the regex understand both so you only need a single one and in that case you keep the same identifier so it overrides the built in indexer definition.

There is a usecase for this (featured torrents, reseed requests, requests etc) but we would need to change quite a few bits to get that working. I'm not opposed to it but it's not something we'll rush into implementing at this time 😄

[zip6como]

You could setup a custom definition and copy paste the existing indexer, set a new identifier + name and modify the irc parse lines to look for the Torrent vorgestellt: lines.
That way you could use it and filter for it explicitly,

I will try that.

There is a usecase for this (featured torrents, reseed requests, requests etc) but we would need to change quite a few bits to get that working. I'm not opposed to it but it's not something we'll rush into implementing at this time 😄

Understandable.

Well thank you very much for your time and help.

[zip6como]

I will try that.

It worked.
Thanks again for your efforts.