Test derivative download on mco-staging

[elynema]

Description

On mco-staging, we have derivatives that point to old file systems as well as MDPI content that is on HCP via S3. These cases are not all present in avalon-dev where we can QA #5746.

Need to test against:

• File system references for derivative files with server host names that don't exist
• File system references for derivative files for current and valid server host names/domains
• HCP - these streaming URLs should all be consistent, but need to verify the structure of the relative file path in the derivative metadata (location URL) for this content

Done Looks Like

• Identify different types of storage situations and records we need to test
  • HCP - MDPI content - https://mco-staging.dlib.indiana.edu/media_objects/kw52jh108.json
  • NFS - content directly uploaded into MCO - https://mco-staging.dlib.indiana.edu/media_objects/8910kg12m
  • NFS - older content that points to old server host name - https://mco-staging.dlib.indiana.edu/media_objects/dz010q26z.json
  • very old piece of content - https://mco-staging.dlib.indiana.edu/media_objects/b8515n60s
  • newly uploaded piece of content (directly uploaded to mco-staging) - https://mco-staging.dlib.indiana.edu/media_objects/7d279k27q
  • content migrated over from MCO with 7.7 upgrade (NFS) - https://mco-staging.dlib.indiana.edu/media_objects/5x21v3034.json
  • content migrated over from MCO with 7.7 upgrade (HCP) - https://mco-staging.dlib.indiana.edu/media_objects/vx022667z - https://mco-staging.dlib.indiana.edu/media_objects/gf06gv30h
  • weird edge case: - https://mco-staging.dlib.indiana.edu/media_objects/qj72p820d.json
• Cherrypick Allow download of derivatives from MCO for staff #5746 to mco-staging and test
• Roll back Allow download of derivatives from MCO for staff #5746 from mco-staging

[elynema]

@joncameron Do we need to work collaboratively to identify types of storage situations and records needed to test, or is that something you can do? We can add this to Swarm if needed.

[joncameron]

Mason or Chris would have a better idea but I can start by adding the cases I'm aware of. Discussing at Swarm seems like a good idea so we can review as a team and get a definitive list.

[elynema]

With current logic, code may not be able to identify the HCP items and know to put a different absolute context on them.

[joncameron]

Finished adding examples from mco-staging for each scenario.

[cjcolvar]

Code has been cherry-picked to mco-staging and the ENCODE_WORK_DIR environment variable has been set to /srv/nfs_avalon/avalon_mco-stage/media matching sidekiq for mco-staging. This may only work for items created on mco-staging and not for MCO items.

[elynema]

Some first pass testing

• HCP - MDPI content - https://mco-staging.dlib.indiana.edu/media_objects/kw52jh108.json - DOES NOT WORK
• NFS - content directly uploaded into MCO - https://mco-staging.dlib.indiana.edu/media_objects/8910kg12m - DOES NOT WORK
• NFS - older content that points to old server host name - https://mco-staging.dlib.indiana.edu/media_objects/dz010q26z.json - DOES NOT WORK
• very old piece of content - https://mco-staging.dlib.indiana.edu/media_objects/b8515n60s - DOES NOT WORK
• content migrated over from MCO with 7.7 upgrade (NFS) - https://mco-staging.dlib.indiana.edu/media_objects/5x21v3034.json - DOES NOT WORK
• content migrated over from MCO with 7.7 upgrade (HCP) - https://mco-staging.dlib.indiana.edu/media_objects/vx022667z - DOES NOT WORK
• https://mco-staging.dlib.indiana.edu/media_objects/gf06gv30h - DOES NOT WORK
• weird edge case: - https://mco-staging.dlib.indiana.edu/media_objects/qj72p820d.json - DOES NOT WORK
• newly uploaded piece of content (directly uploaded to mco-staging) - https://mco-staging.dlib.indiana.edu/media_objects/7d279k27q - WORKS!