Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert longer media varchar fields to text in the catalog database #4312

Closed
AetherUnbound opened this issue May 10, 2024 · 2 comments · Fixed by #4357
Closed

Convert longer media varchar fields to text in the catalog database #4312

AetherUnbound opened this issue May 10, 2024 · 2 comments · Fixed by #4357
Assignees
Labels
🗄️ aspect: data Concerns the data in our catalog and/or databases 🧰 goal: internal improvement Improvement that benefits maintainers, not users 🟧 priority: high Stalls work on the project or its dependents 🧱 stack: catalog Related to the catalog and Airflow DAGs

Comments

@AetherUnbound
Copy link
Contributor

Description

Related #4279, see that issue for the rationale.

Alter the audio and image tables in the catalog to convert the following fields from character varying to text in the underlying database:

  • foreign_identifier
  • foreign_landing_url
  • url
  • thumbnail
  • creator
  • creator_url
  • title
  • audio_set_foreign_identifier (audio only)

This will require crafting the migration steps for the catalog database by hand and applying them, then altering the SQL DDL for the tables to reflect this difference. The alters should look like this:

ALTER TABLE audio ALTER COLUMN title TYPE TEXT USING title :: TEXT;

Postgres uses the same underlying storage methods for both these types, so the conversion from one to the other should be extremely fast.

Note
This is blocked by #4311, as the target columns need to be expanded first before the source ones can, otherwise the data refresh will fail.

@AetherUnbound AetherUnbound added ⛔ status: blocked Blocked & therefore, not ready for work 🗄️ aspect: data Concerns the data in our catalog and/or databases 🟧 priority: high Stalls work on the project or its dependents 🧰 goal: internal improvement Improvement that benefits maintainers, not users 🧱 stack: catalog Related to the catalog and Airflow DAGs labels May 10, 2024
@AetherUnbound AetherUnbound self-assigned this May 10, 2024
@AetherUnbound AetherUnbound removed the ⛔ status: blocked Blocked & therefore, not ready for work label May 13, 2024
@AetherUnbound AetherUnbound removed their assignment May 13, 2024
@sarayourfriend
Copy link
Contributor

@WordPress/openverse-catalog is anyone able to write the migration for this?

@AetherUnbound
Copy link
Contributor Author

I'll pick this up later this week if no one gets to it before me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🗄️ aspect: data Concerns the data in our catalog and/or databases 🧰 goal: internal improvement Improvement that benefits maintainers, not users 🟧 priority: high Stalls work on the project or its dependents 🧱 stack: catalog Related to the catalog and Airflow DAGs
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

2 participants