Active Storage with Direct Upload js giving syntax error

[modusss]

My goal

1 - Starts to upload to the cloud after the moment it is attached on client side (before clicking on 'update' button)

2 - Integrate the upload view with another third party js library (as uppy)

From rails documentation:
https://edgeguides.rubyonrails.org/active_storage_overview.html#direct-uploads

If you want to use the Direct Upload feature from a JavaScript framework, or you want to integrate custom drag and drop solutions, you can use the DirectUpload class for this purpose. Upon receiving a file from your library of choice, instantiate a DirectUpload and call its create method. Create takes a callback to invoke when the upload completes.

import { DirectUpload } from "@rails/activestorage"

const input = document.querySelector('input[type=file]')

// Bind to file drop - use the ondrop on a parent element or use a
//  library like Dropzone
const onDrop = (event) => {
  event.preventDefault()
  const files = event.dataTransfer.files;
  Array.from(files).forEach(file => uploadFile(file))
}

// Bind to normal file selection
input.addEventListener('change', (event) => {
  Array.from(input.files).forEach(file => uploadFile(file))
  // you might clear the selected files from the input
  input.value = null
})

const uploadFile = (file) => {
  // your form needs the file_field direct_upload: true, which
  //  provides data-direct-upload-url
  const url = input.dataset.directUploadUrl
  const upload = new DirectUpload(file, url)

  upload.create((error, blob) => {
    if (error) {
      // Handle the error
    } else {
      // Add an appropriately-named hidden input to the form with a
      //  value of blob.signed_id so that the blob ids will be
      //  transmitted in the normal upload flow
      const hiddenField = document.createElement('input')
      hiddenField.setAttribute("type", "hidden");
      hiddenField.setAttribute("value", blob.signed_id);
      hiddenField.name = input.name
      document.querySelector('form').appendChild(hiddenField)
    }
  })
}

Expected behavior

Load js files normally

Actual behavior

This line of code is:
var upload = new !(function webpackMissingModule() { var e = new Error("Cannot find module '@rails/activestorage'"); e.code = 'MODULE_NOT_FOUND'; throw e; }())(file, url);

My configuration

Rails version: 5.2.3
Ruby 2.4.5
Obs: Active Storage itself is working properly

links

this issue I also posted on stackoverflow:
https://stackoverflow.com/questions/56132477/active-storage-direct-uploader-giving-syntax-error

[JetsonDavis]

+1

[colinrubbert]

@modusss Did you find a solution for this? I've been upgrading my app to 6 rc1 and now I'm running into this issue too.

[JetsonDavis]

Hi Colin,

Glad you posted, I was about to try to upgrade to 6rc1 also to see if it would make my problem go away.

My workaround was to simply put a js listener on the file_field and that did the trick.
$(".submit-btn").click();

The only problem is that the page refreshes. I'd like to access DirectUpload directly, but when I put this in my view...
import { DirectUpload } from "@rails/activestorage"
I get this error:
SyntaxError: import declarations may only appear at top level of a module

If I change the script tag to:
<script type="module">
I get this error:
TypeError: Error resolving module specifier: @rails/activestorage

Were you able to import DirectUpload?

Thx

[colinrubbert]

@JetsonDavis I'm currently pecking away at the issues, seems like some messed up dependency issues currently. At least for me. The upgrade from 5.2 -> 6 rc1, isn't terrible but there's a bit too much cleanup that isn't noted in the documentation and I'm not entirely sure if it's my environment or if it's a universal thing. For now I'm just pecking away at errors and dependencies.

[JetsonDavis]

Ok, so you're not working on getting DirectUpload to work automagically yet? I'll let you know if I figure anything out!

[colinrubbert]

@JetsonDavis @modusss After much messing around, uninstalling/reinstalling node_modules and such. I stumbled upon this little gem environment.loaders.delete('nodeModules') I added that to my config/webpack/environment.js file and it's working now. Absolutely no clue what the heck is up!
My environment.js file looks like this:

const { environment } = require('@rails/webpacker')

module.exports = environment

environment.loaders.delete('nodeModules')

There has to be a bug somewhere but 12 hours into doing this upgrade to webpacker and I'm done dealing with this crap for today, I've wasted too much work hours on it already. Hopefully this helps, not sure that it will though 🤔

[JetsonDavis]

Hey Colin,

Thanks for sharing your findings! I've had success using the asset pipeline to access the DirectUpload object without having to import it. I created a file 'direct_uploads.js' (copied from the Rails guide page), added it to assets/javascripts, and put '/=direct_uploads in the application.js.

Since I'm only using it in one view, I'm considering using something like 'jquery-readyselector' to trigger my ready script only in the appropriate page. Ever tried that?

[jakeNiemiec]

@colinrubbert, here is some more info on that: https://github.com/rails/webpacker/blob/54c3ca9245e9ee330f8ca63b447c202290f7b624/docs/v4-upgrade.md#excluding-node_modules-from-being-transpiled-by-babel-loader

This was a controversial change that fixes things for some but hurts others.

[colinrubbert]

@jakeNiemiec that's precisely where I got the solve from! I'm a bit too in the weeds right now to have remembered where I got it from 🤣

[modusss]

@JetsonDavis @modusss After much messing around, uninstalling/reinstalling node_modules and such. I stumbled upon this little gem environment.loaders.delete('nodeModules') I added that to my config/webpack/environment.js file and it's working now. Absolutely no clue what the heck is up!
My environment.js file looks like this:

const { environment } = require('@rails/webpacker')

module.exports = environment

environment.loaders.delete('nodeModules')

There has to be a bug somewhere but 12 hours into doing this upgrade to webpacker and I'm done dealing with this crap for today, I've wasted too much work hours on it already. Hopefully this helps, not sure that it will though

I tried to add this line and I restarted the server and the error persists.

What is your Rails version ?

[modusss]

Hey Colin,

Thanks for sharing your findings! I've had success using the asset pipeline to access the DirectUpload object without having to import it. I created a file 'direct_uploads.js' (copied from the Rails guide page), added it to assets/javascripts, and put '/=direct_uploads in the application.js.

Since I'm only using it in one view, I'm considering using something like 'jquery-readyselector' to trigger my ready script only in the appropriate page. Ever tried that?

Hi, I also tried to do as you and the problem relies on this part:

import { DirectUpload } from "activestorage"

I guess that this error occurs because of the ES6 is not supported outside the webpack folder ?

[jakeNiemiec]

Try this:
import { DirectUpload } from "activestorage" 👉 import { DirectUpload } from "@rails/activestorage";

You should be getting this which points to this.

[modusss]

Try this:
import { DirectUpload } from "activestorage" import { DirectUpload } from "@rails/activestorage";

You should be getting this which points to this.

In both cases the javascript error is:

Uncaught SyntaxError: Unexpected token {

I created a file "direct_uploader.js" and required in application.js as you told. Don't you use a transpiler gem to use ES6 in this section ?

[colinrubbert]

@modusss Rails 6.0.0.rc1 is my Rails version.

So a few things that I've done is completely delete the node_modules, yarn remove active_storage and webpacker, yarn add active_storage and webpacker, yarn install to rebuild the node_modules, add that code snippet to my environments, much messing around cleaning up sprockets and sorting out the new application.js location with webpacker and I'm sure a half dozen other things (I've put roughly 18 work hours into sorting this out).

It's been incredibly stressful. What I've noticed too is that it works best if you run rails s AND bin/webpack-dev-server so that the dev-server can compile the assets while rails server runs the app. My brain is pretty freaking mush at this point of working on this problem so I'm not entirely sure what was the fix, probably all of it.

The process needs to be worked on.

[JetsonDavis]

@colinrubbert re we talking about the same thing? You are trying to access the DirectUpload js module?
@modusss - I thought I had gotten past this error but am still getting 'SyntaxError: import declarations may only appear at top level of a module' from ' import { DirectUpload } from "@rails/activestorage";'
I also tried to put the import statement in <script> tags in my view and get this:
TypeError: Error resolving module specifier: @rails/activestorage
using both versions suggested by Jake.

[colinrubbert]

@JetsonDavis yes and maybe no ¯_(ツ)_/¯

I had the same DirectUpload error as you guys. Then I fixed that, then I had a ton more errors, then I fixed those and so on and so forth until 18hrs later of trial and error I got everything working properly... so far.

So yes and maybe no ¯_(ツ)_/¯

I sincerely hope that you don't have to traverse the path I have in the last 48hrs so hopefully they're different issues.

[JetsonDavis]

Yeah, I'm just trying to figure out where to put - import { DirectUpload } from 'activestorage!!!!!

[colinrubbert]

@JetsonDavis You shouldn't have put it anywhere, it should automagically be put in the place where it belongs. As far as I can tell all of that stuff should be in your node_modules folder under @rails/activestorage/src/direct_upload.js. What I had found was that webpacker got all wonky and messed up the node_modules stuff so I had to do a lot of remove and rebuild sort of stuff until the files that needed to be in node_modules were actually there, there was a lot missing for whatever reason.

From there your javascript/packs/application.js file should look like this (or similar)

// This file is automatically compiled by Webpack, along with any other files
// present in this directory. You're encouraged to place your actual application logic in
// a relevant structure within app/javascript and only use these pack files to reference
// that code so it'll be compiled.

require("@rails/ujs").start()
require("turbolinks").start()
require("@rails/activestorage").start()
require("channels")

Could you maybe check and verify all of your dependencies and such are actually there. Not sure your versioning either so my experience could be completely different from yours.

[jakeNiemiec]

rails/webpacker#2110 should help solve the initial problem Unexpected token !

You can hotfix by putting this in your environment.js file:

environment.loaders.nodeModules.exclude = /(?:@?babel(?:\/|\\{1,2}|-).+)|regenerator-runtime|core-js/;

[JetsonDavis]

So Colin, a couple of qq's if I may.

@rails is the root of my rails app (which I get with 'which ruby')?

Also, I don't have a '/packs' folder in /javascripts, just 'application.js' which already has '//= require activestorage'. I am using asset pipelne, not npm

[modusss]

With the same app to test I updated Rails version from 5.2.3 -> 6.0.0.rc1 and ruby from 2.4.5 -> 2.6.3p62.

As @colinrubbert posted, the solution to do not have this this error is to add:

config/webpack/environment.js
environment.loaders.delete('nodeModules')

Now I get into the "debugger" that is by default inserted on added on this js section right after we upload a file:

input.addEventListener('change', (event) => {
  debugger
  Array.from(input.files).forEach(file => uploadFile(file))
  // you might clear the selected files from the input
  input.value = null
})

After that I can see on the rails server the image being uploaded as this:

Started POST "/rails/active_storage/direct_uploads" for ::1 at 2019-05-30 16:23:18 -0300
Processing by ActiveStorage::DirectUploadsController#create as JSON
  Parameters: {"blob"=>{"filename"=>"dsc_0372.jpg", "content_type"=>"image/jpeg", "byte_size"=>520866, "checksum"=>"ud1utlVF2VVtsdllwJzWhg=="}, "direct_upload"=>{"blob"=>{"filename"=>"dsc_0372.jpg", "content_type"=>"image/jpeg", "byte_size"=>520866, "checksum"=>"ud1utlVF2VVtsdllwJzWhg=="}}}
   (0.2ms)  begin transaction
  ActiveStorage::Blob Create (1.0ms)  INSERT INTO "active_storage_blobs" ("key", "filename", "content_type", "byte_size", "checksum", "created_at") VALUES (?, ?, ?, ?, ?, ?)  [["key", "mp73gmhx4km1k45doc6di2pev6hd"], ["filename", "dsc_0372.jpg"], ["content_type", "image/jpeg"], ["byte_size", 520866], ["checksum", "ud1utlVF2VVtsdllwJzWhg=="], ["created_at", "2019-05-30 19:23:18.568290"]]
   (85.5ms)  commit transaction
  GCS Storage (5.4ms) Generated URL for file at key: mp73gmhx4vm1k45docgdi2pev6hd (https://storage.googleapis.com/myapiname/mp73gmhx4km1a45doc6diapev6hd?GoogleAccessId=myapiname-889%40cloudimages.iam.gserviceaccount.com&Expires=1559244498&Signature=k%2F1KHgkKW1bDfgMiYnTHztxGGZ1JB9NhR0MIz1vuFvUopISFqU%2BWTCp1llo3tC3SMS6DrsrLQuWu9mJqYnL57Gv9FUgGLcHpANpn1CXRcXLN7XRQTCpyYDgPPzlqikVHyQ8umez0foY7%2FKNvQ8%2BLALsZAtBg0TQKpcplmToGEbfOhqktCcdCMWg3h2fSXq5A9mbC4e6KZYCVZ4fHV2tyqRnMJLVWa%2Bx6%2BzZ3r6nuKBrok7hoQ7d71Y5pywbddKs5aasjN1cbPdKHnnG23qeIZTLFPo0eApP3uracqDDG%2Bqi1s34LQA2jZH7qay%2BN49p4AJU6cvAfnXWeUoJonI%2Bw7Q%3D%3D)
Completed 200 OK in 136ms (Views: 0.7ms | ActiveRecord: 86.6ms | Allocations: 8140)

The problem now to me is that even showing this successful progress through the code in fact this image is not being saved when I click on the button to submit the form and try to see on "show" page.

[colinrubbert]

@JetsonDavis You can just do a rails -v to get the rails version you're using and ruby -v to get current ruby version.

Are you using webpacker at all? Or are you just trying to use ActiveStorage? Also, what exactly are you trying to do? I was working on getting webpacker up and running so that I could use ActionText. I personally didn't have issues with ActiveStorage pre-webpacker conversion.

@modusss Same question, are you solely looking to use ActiveStorage or are you running into the issue on your way to implementing a stable Webpacker environment? I should probably check my image uploads now that I'm on webpacker to make sure those aren't broken, tests did pass before but 🤔

[jakeNiemiec]

@modusss Can you share the POST request from the browser side? You can do this in your dev tools, this is what it looks like in chrome:

Also, the params look out of place:

{
  "blob"=>{
    "filename"=>"dsc_0372.jpg", 
    "content_type"=>"image/jpeg", 
    "byte_size"=>520866, 
    "checksum"=>"ud1utlVF2VVtsdllwJzWhg=="
  }, 
  "direct_upload"=>{
    "blob"=> {
      "filename"=>"dsc_0372.jpg", 
      "content_type"=>"image/jpeg", 
      "byte_size"=>520866, 
      "checksum"=>"ud1utlVF2VVtsdllwJzWhg=="
    }
  }
}

[modusss]

@jakeNiemiec I checked more carefully and I found this error on browser console:

Access to XMLHttpRequest at 'https://storage.googleapis.com/myappl/q0t1dpr6ud5eybxjioda72nwev52?GoogleAccessId=myapp-889%40cloudimages.iam.gserviceaccount.com&Expires=1559249583&Signature=b9NDm2O06B0vZvLj593klNeDLkDOj685x3ityLxi4I%2BsOH25%2Bmif8TXtTpkTkKK72dCthTdmPtk3z%2FP3tnn%2B5cXKQLRncQFi1Y0dvrDMKYSP9fk80jcH0n0fiA68BHplDLqhz85wGP%2FknuFytU0y%2FZP9M%2FNYsAGyOsDkEByybs%2FyNVOBcwxPN2jTNs4ZT4EZK5mgRdgkZpPxlQyUNsT0BscX89JEi%2FQZV0qN9jNAih4o32yqDO8M2KDWJgZkIjYlxulsyGRuHEzVCGY07zfUhQYhYhPHztDJMxd0TbCOj1G0bz5Y2P0uPx%2B%2Bs0Q8McQVJ5bJdVIM8w%2B2704Ai3cH%2Fw%3D%3D' from origin 'http://localhost:3002' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource.

Maybe is something on Google Cloud itself about "CORS policy".

Edit: this CORS problem can be solved from this issue.

@colinrubbert I just want to have active storage to store images on Google Cloud (I currently use Cloudinary).

[JetsonDavis]

@colinrubbert I'm using asset pipeline, no node module. Trying to access the .DirectUpload object from js. The man page says to use import statement but that doesn't work and .DirectUpload calls result in function not found??

I am using the same code @modusss opened this thread with - directly from the man page

[DavidJRobertson]

Today I created a new rails 6.0.0.rc1 app and ran in to a similar issue. I have barely written any code, just started configuring stuff.

On loading any page, I get a delay of a few seconds, and in the logs I get something like this:

Started GET "/experiments" for ::1 at 2019-05-31 00:48:43 +0100
         < removed irrelevant log entries >
[Webpacker] Compiling…
[Webpacker] Compilation failed:

Completed 200 OK in 4725ms (Views: 4717.5ms | ActiveRecord: 5.1ms | Allocations: 16537)

Similarly, running rails webpacker:compile outputs the same 'compilation failed:' message with no details.

However, running bin/webpack gives:

Hash: 62e70d69faa3ff3f7b19
Version: webpack 4.32.2
Time: 3352ms
Built at: 05/31/2019 00:52:42
                                     Asset       Size       Chunks             Chunk Names
    js/application-ab279af6127fc3824d64.js   1.03 MiB  application  [emitted]  application
js/application-ab279af6127fc3824d64.js.map    873 KiB  application  [emitted]  application
      js/hello_erb-92962b35bb61c991d728.js   4.16 KiB    hello_erb  [emitted]  hello_erb
  js/hello_erb-92962b35bb61c991d728.js.map   3.86 KiB    hello_erb  [emitted]  hello_erb
                             manifest.json  689 bytes               [emitted]
Entrypoint application = js/application-ab279af6127fc3824d64.js js/application-ab279af6127fc3824d64.js.map
Entrypoint hello_erb = js/hello_erb-92962b35bb61c991d728.js js/hello_erb-92962b35bb61c991d728.js.map
[./app/javascript/channels sync recursive _channel\.js$] ./app/javascript/channels sync _channel\.js$ 160 bytes {application} [built]
[./app/javascript/channels/index.js] 487 bytes {application} [built]
[./app/javascript/controllers sync recursive _controller\.js$] ./app/javascript/controllers sync _controller\.js$ 186 bytes {application} [built]
[./app/javascript/controllers/hello_controller.js] 2.92 KiB {application} [optional] [built]
[./app/javascript/controllers/index.js] 396 bytes {application} [built]
[./app/javascript/packs/application.js] 822 bytes {application} [built]
[./app/javascript/packs/hello_erb.js.erb] 257 bytes {hello_erb} [built]
[./node_modules/webpack/buildin/amd-options.js] (webpack)/buildin/amd-options.js 80 bytes {application} [built]
[./node_modules/webpack/buildin/global.js] (webpack)/buildin/global.js 1.01 KiB {application} [built]
[./node_modules/webpack/buildin/harmony-module.js] (webpack)/buildin/harmony-module.js 822 bytes {application} [built]
    + 366 hidden modules

ERROR in ./node_modules/@rails/actiontext/app/javascript/actiontext/attachment_upload.js 19:28-40
"export 'DirectUpload' was not found in '@rails/activestorage'
 @ ./node_modules/@rails/actiontext/app/javascript/actiontext/index.js
 @ ./app/javascript/packs/application.js

This error re DirectUpload and actiontext leads me to believe the problem is somehow related to this ticket but I'm not sure what's going on.

--
edit: adding environment.loaders.delete('nodeModules') to my config/webpack/environment.js does resolve this issue but surely this should just work out of the box.

[jakeNiemiec]

adding environment.loaders.delete('nodeModules') to my config/webpack/environment.js does resolve this issue but surely this should just work out of the box.

It should. Can you try with the latest: @rails/webpacker@4.0.6.

should give:

If anyone still gets breakage, please post environment.js, package.json, babel.config.js and application.js to a https://gist.github.com and I will help debug.

[DavidJRobertson]

Upgraded to webpacker 4.0.6 and if I comment out the environment.loaders.delete('nodeModules') line it still fails. Files requested: https://gist.github.com/DavidJRobertson/0229486414cb8d93d7c300770ad3fa6a

[abhaynikam]

The issue with DirectUpload export failing while webpack compile is resolved in here(rails/webpacker#2116) by @jakeNiemiec

Issue is similar to #36368. Unexpected token ! was also resolved too so I think this can be closed.

[JetsonDavis]

My question was never answered but I'm giving up on ActiveStorage and going to try another solution - Shrine (https://shrinerb.com/). Hopefully ActiveStorage will be more robust in r6.

[colinrubbert]

@JetsonDavis Did you happen to try and make a scratch Rails 5.2 or Rails 6 app and then install ActiveStorage. It could be a environment configuration issue. I would recommend doing it with Rails 5.2 because Rails 6 default is Webpacker for asset management. It could help you with your configuration for your main app that you're working on. Do a side by side comparison. ActiveStorage install should do 99% of the heavy lifting for ya.

[JetsonDavis]

Thanks Colin. The other issue with ActiveStorage is that you can't rename the file/blob and/or change the file structure in the AWS bucket to create folders which has also driven me towards another solution (again, unless this has changed in r6?)

[jakeNiemiec]

@DavidJRobertson, can you tell me if 4.0.7 fixes things. Please update your babel.config.js according to these directions.

From your gist, changing this line to false should fix things.

[DavidJRobertson]

@jakeNiemiec that's fixed it, thanks!

[colinrubbert]

@colinrubbert, here is some more info on that: https://github.com/rails/webpacker/blob/54c3ca9245e9ee330f8ca63b447c202290f7b624/docs/v4-upgrade.md#excluding-node_modules-from-being-transpiled-by-babel-loader

This was a controversial change that fixes things for some but hurts others.

@jakeNiemiec Slightly off topic but maybe not. Do you know if this effects production deployments? I deployed my app to heroku for demoing and it's not reading my trix file which is in the node_modules. Just curious if you know if there needs to be a modification for a production environment like heroku.

[jakeNiemiec]

If anyone still gets breakage, please post the full error, environment.js, package.json, babel.config.js and application.js to a https://gist.github.com and I will help debug.

@colinrubbert It can, it just depends on those 4 files and the error. If you tag me in a gist, Ill comment there. Also, make sure you are on webpacker 4.0.7.

[rails-bot]

This issue has been automatically marked as stale because it has not been commented on for at least three months.
The resources of the Rails team are limited, and so we are asking for your help.
If you can still reproduce this error on the 6-0-stable branch or on master, please reply with all of the information you have about it in order to keep the issue open.
Thank you for all your contributions.