Document Google Cloud Storage as s3 media backend

[syphernl]

Is your feature request related to a problem? Please describe.

It would be nice of Google Cloud Storage Buckets could be used for storing media data.

Describe the solution you'd like

A way to use GCS Buckets in HedgeDoc.

Describe alternatives you've considered

• Using a persistent volume to store, but this does not scale well.
• Setting up Minio and proxying traffic to GCS, but this would unnecessarily complicate the stack needed to run HedgeDoc on GCP

Additional context

There is an official Node.js Client library for Google Cloud Storage available which can be used.

[ErikMichelson]

HedgeDoc (1.x + upcoming 2.0) supports an S3 media backend. While S3 can be used with self-hosted tools like MinIO or cloud-services like AWS, most other providers provide a S3 interface too.
Google Cloud Storage can be used via the S3 Endpoint https://storage.googleapis.com.

To avoid having to maintain many different integrations, I don't see the requirement to add an extra GCS client for now.

References:

• https://cloud.google.com/storage/docs/interoperability
• https://dzlab.github.io/gcp/2022/02/26/gs-with-s3-sdk/

[syphernl]

Hi @ErikMichelson

Thanks for your reply!

HedgeDoc (1.x + upcoming 2.0) supports an S3 media backend. While S3 can be used with self-hosted tools like MinIO or cloud-services like AWS, most other providers provide a S3 interface too. Google Cloud Storage can be used via the S3 Endpoint storage.googleapis.com.

I wasn't aware that Google offered an interoperability method for S3. Definitely something worth looking into (and documenting if I manage to get it working)
For some reason I'm now getting an "Access Denied" error when adding an image in a HedgeDoc document whereas it works fine with a simple Python test script that uses boto3 to uploads a file to the bucket using the same credentials.
So it seems that the credentials and S3 interoperability are working fine, except not via HD.

To avoid having to maintain many different integrations, I don't see the requirement to add an extra GCS client for now.

I actually worked on that yesterday after opening this issue. I have a working implementation available for which I could open a PR.
But if it works via the S3 interoperability method it wouldn't require a separate integration.

[syphernl]

Got it to work, I needed the storage.buckets.get permission in order to determine the bucket region.

[ErikMichelson]

Happy to hear that it worked! I'm going to change this issue then towards documenting the usage of GCS with HedgeDoc.

[syphernl]

I written a guide for it but I'm not 100% sure where to place it in the develop branch.
For 1.x there was a page with explanation about normal AWS S3 but for 2.x this page has been removed? Any suggestions as to where I could place it?

[ErikMichelson]

Having a guide to that seems great! As you seemingly tested it with HedgeDoc 1.x, and HedgeDoc 2.0 isn't released yet (still some things to finish up), you could put it into the master branch (docs/content/guides). The docs/mkdocs.yml file also needs a reference to your new guide to include it into the navigation.
For HedgeDoc 2, we'll transfer and (if necessary) adapt the guides later.