[mod] package.html template: additional links (a python dict)

[dkuku]

What does this PR do?

Refactor package template to include custom amount of links
Rename publishedDate variable to published_date
change engines that use this variable

Why is this change important?

How to test this PR locally?

Author's checklist

Related issues

[Bnyro]

Please also update the documentation at https://github.com/searxng/searxng/blob/master/docs/dev/engines/engine_overview.rst and add the new field there too!

[dkuku]

Please also update the documentation at https://github.com/searxng/searxng/blob/master/docs/dev/engines/engine_overview.rst and add the new field there too!

I updated the pr, it's ready for re-review

[Bnyro]

I personally think it would be better to keep source_code_url, documentation_url and homepage normal fields (not part of the dict), because they do behave differently than normal links. It's a good idea to display them together, but for the developer experience, they should stay separated because one would otherwise always need to look up what specific edge cases for the links dictionary exist (as it's not properly documented, and doesn't follow the way other templates work).

The GitHub engine would require modifications too (source_code_url and homepage) looking at the current changes. But when following the suggestion above, it won't need to be modified.

[dkuku]

After thinking about it, I think the easiest way would be to link to the documentation as homepage.
When publishing on hex the documentation is required I think, I can't publish a package when it's not generated.

On the other hand hex does not have a standardized list of keys in the link dictionary and in the example how to publish a package there is only github so this is the only one people add in 95%. But it's not typed and can be written in different casing - some examples I printed out, sometimes there may be links to other packages on github too - when I'm wrapping a rust library so it can be used in elixir I'm adding links to it.

{'Authority (GitHub)': 'https://github.com/infinitered/authority', 'Authority (Hex)': 'https://hex.pm/authority', 'GitHub': 'https://github.com/infinitered/authority_ecto'}
{'Changelog': 'https://github.com/TheFirstAvenger/ecsx_persistence_ecto/blob/master/CHANGELOG.md', 'GitHub': 'https://github.com/TheFirstAvenger/ecsx_persistence_ecto'}
{'Docs': 'https://hexdocs.pm/aws_ex_ray_ecto', 'Github': 'https://github.com/lyokato/aws_ex_ray_ecto'}
{'Docs': 'https://hexdocs.pm/couchx', 'GitHub': 'https://github.com/javierg/couchx'}
{'Documentation': 'https://hexdocs.pm/carguero_ex_audit', 'GitHub': 'https://github.com/carguero/carguero_ex_audit'}
{'GitHub': 'https://github.com/elixir-ecto/ecto'}
{'GitLab': 'https://github.com/tlux/database_yaml_config_provider'}
{'Github': 'https://github.com/sheharyarn/ecto_atomized_map'}
{'Gitlab': 'https://gitlab.com/patatoid/ecto3_mnesia'}
{'Homepage': 'https://buildkite.com/test-analytics', 'Source': 'https://github.com/buildkite/test_collector_elixir'}
{'Homepage': 'https://git.centralstandard.com/open-source/date-params'}
{'github': 'https://github.com/MorphicPro/dissolver'}
{'github': 'https://github.com/rstacruz/authsense'}
{}

[Bnyro]

After thinking about it, I think the easiest way would be to link to the documentation as homepage. When publishing on hex the documentation is required I think, I can't publish a package when it's not generated.

On the other hand hex does not have a standardized list of keys in the link dictionary and in the example how to publish a package there is only github so this is the only one people add in 95%. But it's not typed and can be written in different casing - some examples I printed out, sometimes there may be links to other packages on github too - when I'm wrapping a rust library so it can be used in elixir I'm adding links to it.

{'Authority (GitHub)': 'https://github.com/infinitered/authority', 'Authority (Hex)': 'https://hex.pm/authority', 'GitHub': 'https://github.com/infinitered/authority_ecto'}
{'Changelog': 'https://github.com/TheFirstAvenger/ecsx_persistence_ecto/blob/master/CHANGELOG.md', 'GitHub': 'https://github.com/TheFirstAvenger/ecsx_persistence_ecto'}
{'Docs': 'https://hexdocs.pm/aws_ex_ray_ecto', 'Github': 'https://github.com/lyokato/aws_ex_ray_ecto'}
{'Docs': 'https://hexdocs.pm/couchx', 'GitHub': 'https://github.com/javierg/couchx'}
{'Documentation': 'https://hexdocs.pm/carguero_ex_audit', 'GitHub': 'https://github.com/carguero/carguero_ex_audit'}
{'GitHub': 'https://github.com/elixir-ecto/ecto'}
{'GitLab': 'https://github.com/tlux/database_yaml_config_provider'}
{'Github': 'https://github.com/sheharyarn/ecto_atomized_map'}
{'Gitlab': 'https://gitlab.com/patatoid/ecto3_mnesia'}
{'Homepage': 'https://buildkite.com/test-analytics', 'Source': 'https://github.com/buildkite/test_collector_elixir'}
{'Homepage': 'https://git.centralstandard.com/open-source/date-params'}
{'github': 'https://github.com/MorphicPro/dissolver'}
{'github': 'https://github.com/rstacruz/authsense'}
{}

I think that the links solution you did before was a good solution which can be used for all package links at the hex engine in my opinion - it's probably better than hardcoding the list of git sources (github, gitlab, ...)

I just didn't like the way you handled the translation of the strings for source_code_url and homepage before.

So my take would be - take the commit 8a400c8 as base and just make links work independent from source_code_url and homepage - so that all of these fields can be used as needed by the engines using the template.

[dkuku]

So my take would be - take the commit 8a400c8 as base and just make links work independent from source_code_url and homepage - so that all of these fields can be used as needed by the engines using the template.

The translations could be moved to the engine itself instead of template - then the for loop can take 5 lines.
These are engine specific, for example npm includes bugs link that is hidden currently.

I started to generate the links in a loop because currently it's using multiple if statements to insert the | divider. It's already complicated with 2 links (13 lines).
I wanted to add an optional third link and it became too complicated, so I started to refactor it.

Keeping the existing url's and adding separate logic with for loop would still be a bit ugly and take around 20 lines.

[Bnyro]

The translations could be moved to the engine itself instead of template - then the for loop can take 5 lines.

I think that source code and homepage are available for almost all package engines at SearXNG, so the translation should not be done by each engine individually to reduce boilerplate code.

[Bnyro]

Looks good to me now from the idea, I'm just thinking about whether it would make sense to move the license_url and license_name fields to links too? What's your opinion?

[dkuku]

Looks good to me now from the idea, I'm just thinking about whether it would make sense to move the license_url and license_name fields to links too? What's your opinion?

In hex there is only a list of licenses available so I can't do it in this particular example - It's like tags.
I'm not sure if linking to the actual license text makes sense at all, it may bloat the results a bit. If you are interested in the license then you probably know which one you are interested in.

[Bnyro]

In hex there is only a list of licenses available so I can't do it in this particular example - It's like tags.

Yes, I see, I was rather thinking about the template for packages in general (GitHub, NPM, ...)

[return42]

@dkuku thanks a lot for your work on this topic 👍 .. I'm going to review this PR and #3487 .. both work on packages.html, so I will start with this PR .. right? .. #3487 is approved by @Bnyro so I assume this modification of packages.html is accepted by both of you in sense of #3456 .. which can be closed when this PR (#3464) is merged / right?

In the course of the changes to the PR here, the target, the package.html was taken out in the meantime and the patch history as well as the title of this PR are unclear regarding this target ...

@dkuku is it okay for you if I clean up the commit history and also correct the title of this PR again? (if I have interpreted the course of the PR correctly with my assumptions above).

[dkuku]

@return42 👍🏻 edit both pr-s it too your liking and merge it when done.

[return42]

@dkuku @Bnyro I squashed the commit history into one commit and slightly changed the commit message. And I did some small modifications / see diff

See my comments in this review below for more details .. if you are OK with what I have done, give a short feedback to merge this PR.

Again, thanks a lot for your work on this PR 👍

[dkuku]

Looks good to me.