Solution to source filter list size

[sarayourfriend]

We've started this same discussion half a dozen times, and I can't find a single GitHub issue or the discussion I thought existed about it. This is a big problem with Smithsonian in particular, because the source names are long. The filters list is unusable.

In the screenshot above, Smithsonian sources take up almost the entire screen (not a small screen space!).

With Smithsonian, they are under a single heading, so it makes some sense to collapse them in a heirarchy, like a tree list. As in, we can easily communicate why those sources would be under that heading.

It is clear why these sources are grouped under Smithsonian.

This, on the other hand, is somewhat confusing information. None of these organisations are related to Flickr, they just use it as a source. Some of them are highly prestigious sources, that we would be remiss to collapse under a heading. The same is true of Europeana sources, but with the added complexitiy of the fact that those sources only have data provided by Europeana, and do not host their images there. The relationship is therefore even harder to guess, even if you do click through to the foreign landing URL of a work from that source.

In any case, we need some solution to this problem, because the current approach is untenable, and makes browsing the image sources a pain, especially if you are looking for a particular source, and have to scroll a lot to find it.

Some ideas that could work:

1. Conditionally collapsing sources, (do it for Smithsonian, but not others)
2. Sort sources by the number of results based on the other currently applied filters (including query). We can do this using an Elasticsearch aggregation: https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-terms-aggregation.html. It might be prohibitively computationally expensive. Sort the sources by the number of available results for the current query, rather than alphabetically, so that sources that might actually be useful (rather than disappointing) float to the top, and anything with very few or no results does not clutter the top of the list for that query.
3. Collapse all sources under their provider, while doing something to communicate what provider means in each context (do they host the data, or just provide the data, or are they an umbrella institution underwhich the others are housed; flickr, Europeana, and Smithsonian, respectively)

I like doing both of the first two, and then only the third one if for some reason those first two weren't sufficient. Of course, any heirarchy of sources makes sorting them by available results much tricker! Especially with the third option. Might make sense, for example, to have Smithsonian sources all count towards the provider heading. That doesn't make any sense for Europeana and Flickr, though, for similar reasons to what I've shared above.

Anyway, just want to collect this conversation into a single place so we can finally make a decision and do something instead of starting and restarting this discussion another half dozen times without actually recording a decision.

[obulat]

Thank you for opening this discussion and collecting the conversation into a single place!

We could also group the sources by "type": museums and cultural institutions, scientific collections (e.g., Science museum, iNaturalist, maybe Geograph photos), digital photo collections (a group that would include such sources as Flickr and RawPixel) and collapse them. This could be subjective, but as a user, I would probably like this categorization as it would give me some idea of the type of images I would get.

The second idea seems prohibitively computationally expensive with the scale of our data.

[sarayourfriend]

The second idea seems prohibitively computationally expensive with the scale of our data.

I think ES aggregations can be configured to use statistical sampling rather than actually checking all the data. Kibana, for example, does this by default when you evaluate data from ES, precisely to prevent huge indexes from slowing to a crawl.

The type is great, but depends on #661 and the related issues. That is one of the projects for 2024 we've planned, but I still don't think it solves the huge list of Smithsonian sources, so doesn't address the entire issue.

[fcoveram]

While I was writing my response with interaction details, I decided to test an idea I had with the following features:

• Source search. Narrow the list by searching within the section.
• Sort by: Alphabetical, Category, Results. As current, alphabetical order is the default.
• Number of results. Every source has the results number on the right. To make this work in accordance to the rest of filters, I envision every filter should have this pill.

CleanShot.2024-02-22.at.15.57.10.mp4

The above is an early design idea that needs plenty of revision of component consistency and a11y. I did not add the expand/collapse action as I see value on keeping all options visible, but I do agree that current list is massive and needs better affordance for users.

---

On the other hand, I checked the official site of every Smithsonian Institution source and almost all of them don't have the Smithsonian Institution: text in their names. The only special cases I found are:

1. Smithsonian Institution: Freer Gallery of Art → The official site refers to the organization as "Freer Gallery of Art," but it highlights that is part of The Smithsonian's National Museum of Asian Art.
2. Smithsonian Institution: National Zoo → The official site shows the name "Smithsonian's National Zoo and Conservation Biology Institute" in the logo and other texts.

From the above, I think we can safely delete the Smithsonian Institution: part in the rest of sources and keep the discussion on what to do with the two cases mentioned above.

[AetherUnbound]

This is so cool!! I love the design 🤩

[AetherUnbound]

Thanks for aggregating this discussion Sara! As an easy approach, I do really like the idea of conditionally aggregating and collapsing sources like the Smithsonian sources. Perhaps this could be a change we could make on the API end so that we could configure the groups using the Django Admin UI?

[sarayourfriend]

As it's a presentational thing, I'm inclined to think it should be purely the responsibility of the frontend, unless it's something we think other integrations should also consider. If we did it API side we could update the stats endpoint to return a parent in each source or to actually nest them in an arrray of children.

However, both of those lock us into public API contracts (unless we keep them indefinitely unstable). Implementing this purely on the frontend avoids that issue as well.

[AetherUnbound]

Ah, I really hadn't considered having this implemented exclusively on the frontend! That makes sense for the reasons you describe.

[AetherUnbound]

Synthesizing some points that were discussed in a team meeting a few weeks ago (2024-02-20), I think we came away with a few points:

• We can shorten the Smithsonian-related museum names (per Francisco's analysis above) and collapse them under a single header
• We can potentially add a search for the sources filtering to make that easier
• We'll wait until Expand Content Providers Metadata #3583 to try and dive into how to handle categories themselves

I'll make issues for the first two points above with that in mind.

[AetherUnbound]

I've made the following:

• Group Smithsonian institutions in the frontend source filter list #3922
• Shorten Smithsonian institution names #3923
• Design source filter list search feature #3924