Uploading image from online link to supabase always returns application/octet-stream

[leadream]

I tried to upload a file from a URL to Supabase storage. I always get an object with application/octet-stream mime type. It only happens on Cloudflare workers. I tried it on a normal server and it works well. Here is my code:

const { createClient } = require('@supabase/supabase-js')

const supabase = createClient('SUPABASE_URL', 'SUPABASE_ANON_KEY')

async function upload (url) {
    const blob = await fetch(url)
        .then(response => response.blob())
    const fileName = 'test.' +blob.type.split('/')[1]
    const result = await supabase
      .storage
      .from('avatars')
      .upload(fileName, blob, {
        cacheControl: '3600',
        contentType: blob.type,
        upsert: true,
      })
      console.log(result)
}

upload('https://tinypng.com/images/panda-happy-2x.png')

[petebacondarwin]

This looks like a Workers runtime related issue, rather than a Wrangler specific one.
I'll escalate to the runtime team to find out more...

[evanderkoogh]

Hey @leadream, this is indeed an interesting problem and most likely with the Worker Runtime.

Now do you have any compatibility date set in your wrangler.toml and if yes, what is it set to?

[leadream]

@evanderkoogh Yes, I have this in my wrangler.toml:

[petebacondarwin]

@leadream can you check whether blob.type after the fetch() is what you expect?

[leadream]

@petebacondarwin Sure, it's what I expect.

My code is:

export async function compressImage(request: Request, env: any, supabase: SupabaseClient): Promise<Response> {
  try {
    const blob = await fetch('https://tinypng.com/images/panda-happy-2x.png')
        .then(response => response.blob())
    console.log('blob type', blob.type)
    const fileName = 'test.' +blob.type.split('/')[1]
    const result = await supabase
      .storage
      .from('avatars')
      .upload(fileName, blob, {
        cacheControl: '3600',
        contentType: blob.type,
        upsert: true,
      })
      console.log(result)
    return handleResponse(result, null)
  } catch (error) {
    return handleResponse(null, error)
  }
}

[petebacondarwin]

I note that you are running this in wrangler dev --local mode. This means that the Worker is being emulated locally in the Miniflare library. Do you get the same results when running in remote mode or when publishing your worker to Cloudflare?

[leadream]

@petebacondarwin I published it and it works well.

[evanderkoogh]

Thanks for that!

So I think we can narrow it down to it being a Miniflare issue then.. @petebacondarwin you are probably better placed to make sure it gets on the agenda over there?

[petebacondarwin]

I think we need to raise an issue here https://github.com/cloudflare/miniflare/issues.
It would be helpful if we could sniff the HTTP requests that are being sent from Miniflare to Supabase and see where it is missing something.
For example I see that @evanderkoogh opened this issue a while ago: cloudflare/miniflare#180
I wonder if there is something similar happening here?

[petebacondarwin]

@mrbbot - any ideas here?

[mrbbot]

Did some digging, this is the code in @supabase/supabase-js that does the uploading: https://github.com/supabase/storage-js/blob/1a7d98c3e8f90c43ab7c7c2b52a34f99bb0736b7/src/lib/StorageFileApi.ts#L74-L76. Looks like undici's FormData class discards Blob types when appending to FormData:

// Local
import { FormData } from "undici";

const blob = new Blob(["test"], { type: "text/plain" });

const formData = new FormData();
formData.append("file", blob);

const formDataEdBlob = formData.get("file");
console.log(formDataEdBlob.type); // ""

// Workers Runtime (and also Firefox)
const blob = new Blob(["test"], { type: "text/plain" });

const formData = new FormData();
formData.append("file", blob);

const formDataEdBlob = formData.get("file");
console.log(formDataEdBlob.type); // "text/plain"

See the corresponding undici code here: https://github.com/nodejs/undici/blob/4684a1543d87e98b441959d731a3a13d20eaa17d/lib/fetch/formdata.js#L244-L248. Probably worth an issue in undici?

---

A potential temporary solution though is to pass a non-Blob/File type to upload, triggering this code path instead: https://github.com/supabase/storage-js/blob/1a7d98c3e8f90c43ab7c7c2b52a34f99bb0736b7/src/lib/StorageFileApi.ts#L81-L83:

const buffer = await blob.arrayBuffer();
const result = await supabase.storage
  .from("avatars")
  .upload(fileName, buffer, {
    cacheControl: "3600",
    contentType: blob.type,
    upsert: true,
  });

[petebacondarwin]

Great investigation @mrbbot! And thanks for the workaround.

I proposed a fix to the undici project here: nodejs/undici#1467

@mrbbot - does the update to a new version of undici need to be tracked here or in Miniflare?

[mrbbot]

I'd say in Miniflare. There's actually an open issue (cloudflare/miniflare#220) to upgrade Miniflare's undici to 4.15.0, so we can just tag it onto that.

[petebacondarwin]

Cool. In that case I am going to close this, since there is nothing to track here and you have provided a workaround.