3384 – update rails to ~> 6.1.7

[vasconsaurus]

Description

There’s a CVE for actionpack in Pender that should be resolved by us upgrading Rails to 6.1.74 (or just doing ~>6.1.7 and upgrading)

References: cv2-3384

How has this been tested?

Ran a request to pender, like the one below, got the parsed response 🎉

curl -X GET -H 'X-Pender-Token: dev' -H 'Content-type: application/json' http://localhost:3200/api/medias.json?url=https://www.tiktok.com/@flowerboyjoy&refresh=1

Things to pay attention to during code review

Although a tiny change, a few things broke:

• Sidekiq: it was initially bumped to 7, but since that one doesn't support delayed extensions anymore, I kept it under that. However, we use that in one place, it doesn't look like it will be super hard to upgrade(🤞). Seems to be setup here (03_sidekiq.rb#L27) and used here(media_archive_org_archiver.rb#L28).
• Addressable/Postrank: kept addressable at 2.8.1, after that version there is an issue with Postrank, and we are not able to run uri.normalize

relevant links / sidekiq's upgrade

• https://github.com/sidekiq/sidekiq/blob/main/Changes.md#640
• Remove delayed extensions sidekiq/sidekiq#5076
• https://github.com/sidekiq/sidekiq/blob/main/docs/7.0-Upgrade.md
• https://github.com/sidekiq/sidekiq/blob/main/docs/7.0-API-Migration.md

relevant links / addressable issue

• Normalize errors when trying to run a simple url normalize sporkmonger/addressable#513
• Any version after 2.8.1 causes errors in our test suite coming from addressable. sporkmonger/addressable#506
• Remove Addressable monkey patch? postrank-labs/postrank-uri#49

[vasconsaurus]

@hartsick I did the bundle update, seems like there wasn't javascript stuff? Though I'm really not sure if I'm missing something. But the build errored, I'll try to look into this again later.

[hartsick]

@vasconsaurus Yep! Bundle just updates Ruby dependencies. If we need Javascript updates you'll still need to run the Javascript commands you mentioned in the ticket.

Also, sorry for delay on response - had this typed out last week and forgot to press comment apparently!

[vasconsaurus]

@hartsick sorry I misspoke, I meant it doesn't seem like there is any javascript stuff to update, but I'm not sure.
And the build is erroring pretty much at the beginning, so I'm sure I'm missing something.

[hartsick]

@vasconsaurus no worries! were you able to build the docker image and spin up the container locally? wondering if we can reproduce the problem outside of CI

[vasconsaurus]

@hartsick ✨ Updates ✨ (I think I'm going to add this to the ticket as well)

• figured out what was wrong, when we updated rails, sidekiq was bumped to 7, and that version doesn't have delayed extensions anymore. So I kept it under 7 for now.
• that said we pretty much use that in one place, I tested removing it from the config and the file using it, and it 'worked' (well, docker compose did), so we would only need to update those two things
• however something else is breaking, I started getting errors saying the url was invalid. From poking around it seems the problem is in RequestHelper.parse_url. When I try to normalize the uri it fails

(byebug) uri
#<Addressable::URI:0x5780 URI:https://rubydoc.info/gems/addressable>
(byebug) uri.normalize
*** TypeError Exception: Can't convert Object into String.

• uri has the methods normalize, normalize!. I couldn't figure out what is happening. But since we get that error when trying to normalize it, it returns the UrlFormatError

[codeclimate]

Code Climate has analyzed commit 6af1098 and detected 0 issues on this pull request.

The test coverage on the diff in this pull request is 100.0% (50% is the threshold).

This pull request will bring the total coverage in the repository to 98.0% (0.0% change).

View more on Code Climate.

[caiosba]

Good to go if tests pass! I didn't see any major upgrade of concern (like, Sidekiq would be one, so I'm glad we're guarding against that).

[vasconsaurus]

ran integration tests, they passed 🎉 so I'm merging this one

@caiosba should we open a ticket to do the sidekiq upgrade?

[caiosba]

@vasconsaurus sure, but no need to schedule it yet. Thanks!