Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Duplicate logic for formatting human readable file sizes #3419

Open
Bnyro opened this issue Apr 25, 2024 · 1 comment
Open

Duplicate logic for formatting human readable file sizes #3419

Bnyro opened this issue Apr 25, 2024 · 1 comment
Labels
bug Something isn't working

Comments

@Bnyro
Copy link
Member

Bnyro commented Apr 25, 2024

@return42 added a humanize_bytes function in e76ab1a.

However I figured out when working on a PR recently, that we already had a similar logic before https://github.com/searxng/searxng/blob/master/searx/templates/simple/result_templates/files.html#L38 used in the files.html template - it however covers less cases that the humanize_bytes function.

We should probably replace the logic in files.html with using humanize_bytes.

@Bnyro Bnyro added the bug Something isn't working label Apr 25, 2024
@return42
Copy link
Member

Yeah, good point .. we should normalize all the implementation to ..

searxng/searx/utils.py

Lines 355 to 364 in e021441

def humanize_bytes(size, precision=2):
"""Determine the *human readable* value of bytes on 1024 base (1KB=1024B)."""
s = ['B ', 'KB', 'MB', 'GB', 'TB']
x = len(s)
p = 0
while size > 1024 and p < x:
p += 1
size = size / 1024.0
return "%.*f %s" % (precision, size, s[p])

You already mentioned the template ..

{%- if result.size %}<tr><td>{{ _('Filesize') }}</td><td>
{%- if result.size < 1024 %}{{ result.size }} {{ _('Bytes') -}}
{%- elif result.size < 1024*1024 %}{{ '{0:0.2f}'.format(result.size/1024) }} {{ _('kiB') -}}
{%- elif result.size < 1024*1024*1024 %}{{ '{0:0.2f}'.format(result.size/1024/1024) }} {{ _('MiB') -}}
{%- elif result.size < 1024*1024*1024*1024 %}{{ '{0:0.2f}'.format(result.size/1024/1024/1024) }} {{ _('GiB') -}}
{%- else %}{{ '{0:0.2f}'.format(result.size/1024/1024/1024/1024) }} {{ _('TiB') }}{% endif -%}
</td></tr>
{%- endif -%}

Haven't look deep, but it seems to me the torrent engines first do calculate from a human readable format to the number of bytes (see utils.get_torrent_size(..)) and in the file template of the torrent engine the number of bytes is calculated back to a human readable format .. may this can be also be simplified a little bit(?).

./searx/engines/btdigg.py:66:        filesize = get_torrent_size(filesize, filesize_multiplier)
./searx/engines/tokyotoshokan.py:78:                    params['filesize'] = get_torrent_size(groups[0], groups[1])
./searx/engines/1337x.py:45:        filesize = get_torrent_size(filesize, filesize_multiplier)
./searx/engines/bt4g.py:117:                'filesize': get_torrent_size(filesizeParsed[0], filesizeParsed[1]),
./searx/engines/piratebay.py:90:            filesize = get_torrent_size(result["size"], "B")
./searx/engines/digbt.py\08:from searx.utils import extract_text, get_torrent_size
./searx/engines/digbt.py:48:        filesize = get_torrent_size(files_data[FILESIZE], files_data[FILESIZE_MULTIPLIER])
./searx/engines/solidtorrents.py:17:    get_torrent_size,
./searx/engines/solidtorrents.py:66:            'filesize': get_torrent_size(*extract_text(stats[1]).split()),
./searx/engines/nyaa.py:106:            filesize = get_torrent_size(*filesize_info.split())
./searx/engines/kickass.py:57:        result['filesize'] = get_torrent_size(*extract_text(eval_xpath(tag, './/td[contains(@class, "nobr")]')).split())

searxng/searx/utils.py

Lines 332 to 352 in e021441

def get_torrent_size(filesize: str, filesize_multiplier: str) -> Optional[int]:
"""
Args:
* filesize (str): size
* filesize_multiplier (str): TB, GB, .... TiB, GiB...
Returns:
* int: number of bytes
Example:
>>> get_torrent_size('5', 'GB')
5368709120
>>> get_torrent_size('3.14', 'MiB')
3140000
"""
try:
multiplier = _STORAGE_UNIT_VALUE.get(filesize_multiplier, 1)
return int(float(filesize) * multiplier)
except ValueError:
return None

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants