Download multiple structures at once

[CasperWA]

A button to download all found structures would be nice to have.

This would need to be implemented in a non-blocking way.

[csadorf]

Is this in compliance with all data base provider ToS?

[CasperWA]

Is this in compliance with all data base provider ToS?

Concerning data amounts, they set their own limit of how many structures may be present in a single response.

Conerning downloading their structures, rights, etc., there is no difference from what we're currently doing.

The only issue that may arrive (that I can think of) is if the requested amount of structures to download is extreme. However, e.g., for Materials Project and COD it is not uncommon that users download extremely large chunks of their database several times daily (afaik, @merkys and @shyamd could comment on this if they want).

If the database is not an openly accessible database this is different and something we still haven't taken into account on the side of OPTIMADE. Indeed, the consortium's current solution to this is to let the providers set this up and manage it themselves (authentication, etc.). Again, I would probably also expect the provider to set up limits to the amount of requests that can be made as well as how many structures are allowed per response (i.e., per "page").

[merkys]

The only issue that may arrive (that I can think of) is if the requested amount of structures to download is extreme. However, e.g., for Materials Project and COD it is not uncommon that users download extremely large chunks of their database several times daily (afaik, @merkys and @shyamd could comment on this if they want).

Indeed. While download of files in CIF format is cheap, as this is plain file transfer, serving large amounts of structures via OPTIMADE API is expensive, as currently symmetry reconstruction is done on-the-fly in the COD implementation.

Again, I would probably also expect the provider to set up limits to the amount of requests that can be made as well as how many structures are allowed per response (i.e., per "page").

Provisions for such limits are already in OPTIMADE specification, however, the actual numbers have to be determined in trial-and-failure manner, as no means are present to communicate this number. Would be a nice feature request.

[shyamd]

Normally, we have API keys to ensure this doesn't happen as random attacks. WIth the OPTIMADE API as implemented pagination + rate-limiting should prevent anyone from really killing our API. In the future, we might go back to having an API key to protect this endpoint. I don't know how you will want to deal with that in your Voila client.